Jul 06 14:12:54.537209 ip-10-0-131-65 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 06 14:12:54.537223 ip-10-0-131-65 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 06 14:12:54.537233 ip-10-0-131-65 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 06 14:12:54.537560 ip-10-0-131-65 systemd[1]: Failed to start Kubernetes Kubelet. Jul 06 14:13:04.588537 ip-10-0-131-65 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 06 14:13:04.588554 ip-10-0-131-65 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 1a003cf082434027b4f2e0f8d8be75de -- Jul 06 14:14:44.203488 ip-10-0-131-65 systemd[1]: Starting Kubernetes Kubelet... Jul 06 14:14:44.541157 ip-10-0-131-65 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:14:44.541157 ip-10-0-131-65 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 14:14:44.541157 ip-10-0-131-65 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:14:44.541157 ip-10-0-131-65 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 14:14:44.541157 ip-10-0-131-65 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 14:14:44.543114 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.542999 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 14:14:44.548080 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548047 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.548080 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548074 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.548080 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548079 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.548080 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548083 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.548080 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548087 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548092 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548096 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548099 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548103 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548110 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548115 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548119 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548124 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548128 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548132 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548136 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548140 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548144 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548148 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548152 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548156 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548160 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548166 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.548409 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548172 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548176 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548182 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548187 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548212 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548221 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548225 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548229 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548233 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548237 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548240 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548244 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548247 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548251 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548255 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548260 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548265 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548270 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548274 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.549168 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548278 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548282 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548286 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548290 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548295 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548299 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548303 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548307 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548311 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548315 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548320 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548324 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548328 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548332 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548336 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548340 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548344 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548349 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548353 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548357 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.549967 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548361 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548365 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548369 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548373 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548379 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548384 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548388 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548392 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548398 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548402 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548406 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548410 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548414 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548418 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548421 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548425 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548430 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548434 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548438 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.550811 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548442 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548446 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548452 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548456 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548460 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548464 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548469 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548473 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548477 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548480 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548484 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548488 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548492 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548497 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548501 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548505 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548509 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.548514 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549715 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549725 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549730 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.551644 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549735 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549740 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549744 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549749 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549753 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549757 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549761 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549766 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549770 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549774 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549778 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549782 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549786 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549791 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549798 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549805 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549812 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549817 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549821 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.552605 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549826 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549829 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549834 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549838 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549842 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549846 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549850 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549854 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549858 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549862 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549867 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549873 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549879 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549884 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549889 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549893 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549898 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549902 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549906 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549910 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.553170 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549914 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549918 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549922 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549928 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549934 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549938 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549943 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549947 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549953 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549957 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549961 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549965 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549969 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549972 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549976 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549980 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549984 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549988 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549992 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549995 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.553763 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.549999 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550003 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550007 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550011 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550017 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550022 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550026 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550030 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550034 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550039 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550043 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550048 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550052 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550056 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550060 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550064 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550068 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550073 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550077 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550081 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.554407 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550087 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550092 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550097 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550101 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550105 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550109 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550113 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550117 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550121 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550125 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550129 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550133 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550137 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550142 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550146 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550150 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550154 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550780 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550788 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550792 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.555004 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550796 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550800 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550805 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550809 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550813 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550817 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550821 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550825 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550829 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550833 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550837 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550841 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550845 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550850 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550854 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550858 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550862 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550866 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550870 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550875 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.555533 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550878 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550883 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550887 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550891 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550895 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550899 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550903 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550907 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550912 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550916 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550920 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550925 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550929 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550935 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550941 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550946 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550951 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550956 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550960 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.556118 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550964 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550968 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550973 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550978 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550983 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550987 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550991 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.550997 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551001 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551005 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551010 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551016 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551021 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551025 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551030 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551034 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551038 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551042 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551046 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551050 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.556626 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551054 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551058 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551063 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551067 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551071 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551075 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551079 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551083 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551087 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551091 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551096 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551100 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551104 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551113 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551118 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551123 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551127 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551132 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551136 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.557108 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551140 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551145 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551150 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551154 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551158 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551162 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551166 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551170 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551175 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551179 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551183 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551187 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551210 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551214 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551218 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551222 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551226 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.551231 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551334 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551346 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551356 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 14:14:44.557592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551364 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551371 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551376 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551383 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551389 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551394 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551399 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551404 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551409 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551414 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551418 2581 flags.go:64] FLAG: --cgroup-root="" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551423 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551428 2581 flags.go:64] FLAG: --client-ca-file="" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551437 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551441 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551448 2581 flags.go:64] FLAG: --cluster-domain="" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551454 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551459 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551464 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551469 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551476 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551480 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551485 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 14:14:44.558249 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551490 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551495 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551500 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551504 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551509 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551514 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551521 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551526 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551530 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551535 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551540 2581 flags.go:64] FLAG: --enable-server="true" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551544 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551552 2581 flags.go:64] FLAG: --event-burst="100" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551557 2581 flags.go:64] FLAG: --event-qps="50" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551561 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551567 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551571 2581 flags.go:64] FLAG: --eviction-hard="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551577 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551582 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551587 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551591 2581 flags.go:64] FLAG: --eviction-soft="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551596 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551602 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551607 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551612 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 14:14:44.558798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551618 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551622 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551627 2581 flags.go:64] FLAG: --feature-gates="" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551633 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551638 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551643 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551647 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551652 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551657 2581 flags.go:64] FLAG: --help="false" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551661 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551666 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551670 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551675 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551680 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551686 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551690 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551695 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551699 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551704 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551708 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551713 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551718 2581 flags.go:64] FLAG: --kube-reserved="" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551722 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551727 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 14:14:44.559505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551732 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551736 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551741 2581 flags.go:64] FLAG: --lock-file="" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551745 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551750 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551758 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551768 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551774 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551780 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551785 2581 flags.go:64] FLAG: --logging-format="text" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551789 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551795 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551799 2581 flags.go:64] FLAG: --manifest-url="" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551804 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551811 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551816 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551822 2581 flags.go:64] FLAG: --max-pods="110" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551826 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551831 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551835 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551840 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551844 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551849 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551854 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551865 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 14:14:44.560089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551870 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551874 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551879 2581 flags.go:64] FLAG: --pod-cidr="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551883 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551892 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551896 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551901 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551905 2581 flags.go:64] FLAG: --port="10250" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551910 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551915 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a939b040dc12f733" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551920 2581 flags.go:64] FLAG: --qos-reserved="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551924 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551929 2581 flags.go:64] FLAG: --register-node="true" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551935 2581 flags.go:64] FLAG: --register-with-taints="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551941 2581 flags.go:64] FLAG: --registry-burst="10" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551948 2581 flags.go:64] FLAG: --registry-qps="5" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551952 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551957 2581 flags.go:64] FLAG: --reserved-memory="" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551974 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551979 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551984 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551988 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551993 2581 flags.go:64] FLAG: --runonce="false" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.551997 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552002 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 14:14:44.560779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552006 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552010 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552015 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552020 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552024 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552029 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552034 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552038 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552043 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552047 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552052 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552057 2581 flags.go:64] FLAG: --system-cgroups="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552061 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552070 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552074 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552079 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552087 2581 flags.go:64] FLAG: --tls-min-version="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552092 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552096 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552101 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552107 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552111 2581 flags.go:64] FLAG: --v="2" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552126 2581 flags.go:64] FLAG: --version="false" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552134 2581 flags.go:64] FLAG: --vmodule="" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552140 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 14:14:44.561397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.552145 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552346 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552355 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552360 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552365 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552369 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552374 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552379 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552383 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552387 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552391 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552395 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552399 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552403 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552407 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552411 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552415 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552419 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552424 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552428 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.562038 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552432 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552436 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552440 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552444 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552447 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552459 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552463 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552469 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552473 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552479 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552485 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552491 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552496 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552501 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552505 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552510 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552515 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552519 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552523 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.562546 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552527 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552531 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552535 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552539 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552543 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552547 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552551 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552555 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552559 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552565 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552571 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552576 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552580 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552586 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552592 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552596 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552600 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552604 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552608 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.563016 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552612 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552618 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552622 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552628 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552632 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552635 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552641 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552645 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552649 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552654 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552658 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552662 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552666 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552670 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552674 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552678 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552682 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552687 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552691 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552695 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.563497 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552699 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552703 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552707 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552711 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552715 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552719 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552723 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552727 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552731 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552735 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552739 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552744 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552748 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552754 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552758 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552764 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552768 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552772 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552776 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552781 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.563995 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552785 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.552790 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.553466 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561009 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561026 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561046 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561052 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561102 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561107 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561110 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561113 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561115 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561118 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561121 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561124 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.564504 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561127 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561130 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561133 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561136 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561139 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561141 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561143 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561146 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561150 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561153 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561156 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561159 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561163 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561166 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561169 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561172 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561175 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561178 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561180 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.564872 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561183 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561186 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561207 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561212 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561216 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561220 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561222 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561225 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561228 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561230 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561233 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561235 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561238 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561240 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561243 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561245 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561247 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561250 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561252 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.565386 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561255 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561257 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561260 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561262 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561265 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561268 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561272 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561274 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561277 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561280 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561283 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561286 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561289 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561292 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561295 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561298 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561301 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561303 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561306 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.565837 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561308 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561311 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561313 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561316 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561319 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561321 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561324 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561326 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561329 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561331 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561333 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561336 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561338 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561341 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561343 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561346 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561349 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561351 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561353 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561356 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.566382 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561358 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561361 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561363 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561365 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561368 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561370 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561373 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561375 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561378 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561381 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561383 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561385 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561388 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561390 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561395 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:14:44.566852 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561499 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561503 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561506 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561508 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561511 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561513 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561516 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561518 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561521 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561523 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561526 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561528 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561530 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561533 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561535 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561538 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561540 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561544 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561548 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561550 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 14:14:44.567234 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561552 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561555 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561558 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561561 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561563 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561566 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561569 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561571 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561573 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561577 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561580 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561583 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561585 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561588 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561590 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561592 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561595 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561597 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561599 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561602 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 14:14:44.567723 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561604 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561607 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561609 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561612 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561614 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561616 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561619 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561621 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561623 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561626 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561628 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561632 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561635 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561638 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561641 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561643 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561646 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561649 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561651 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561654 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 14:14:44.568240 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561656 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561659 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561661 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561664 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561666 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561668 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561671 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561673 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561676 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561678 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561681 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561683 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561685 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561688 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561690 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561693 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561695 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561697 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561700 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561702 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 14:14:44.568759 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561704 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561707 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561709 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561712 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561714 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561716 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561718 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561721 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561723 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561727 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561729 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561732 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561734 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561737 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561739 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561741 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561744 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561746 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 14:14:44.569437 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:44.561748 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561753 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.561870 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.563780 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.564971 2581 server.go:1019] "Starting client certificate rotation" Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.565075 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.565108 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 14:14:44.569915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.567529 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:14:44.572987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.572965 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 14:14:44.583701 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.583678 2581 log.go:25] "Validated CRI v1 runtime API" Jul 06 14:14:44.587419 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.587404 2581 log.go:25] "Validated CRI v1 image API" Jul 06 14:14:44.588238 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.588214 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 14:14:44.592888 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.592860 2581 fs.go:135] Filesystem UUIDs: map[6bd5ae0d-9a1b-4e02-99b8-6cefc15cbf8a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 84825612-e1d6-4a0c-a825-7fafbeb4f58c:/dev/nvme0n1p3] Jul 06 14:14:44.592967 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.592886 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 14:14:44.598566 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.598448 2581 manager.go:216] Machine: {Timestamp:2026-07-06 14:14:44.597058938 +0000 UTC m=+0.291899176 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098454 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec284dbe0340f54cb13267dff602855f SystemUUID:ec284dbe-0340-f54c-b132-67dff602855f BootID:1a003cf0-8243-4027-b4f2-e0f8d8be75de Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:29:27:15:a4:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:29:27:15:a4:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:c1:30:47:94:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 14:14:44.598566 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.598556 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 14:14:44.598712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.598641 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 14:14:44.602286 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.602256 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 14:14:44.602454 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.602290 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-65.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 14:14:44.603335 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.603321 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:14:44.603733 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.603719 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 14:14:44.603796 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.603741 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 14:14:44.603796 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.603753 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:14:44.604567 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.604555 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 14:14:44.606734 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.606721 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 14:14:44.610162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610148 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:14:44.610321 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610310 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 14:14:44.610376 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610352 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 06 14:14:44.610376 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610367 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 14:14:44.610519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610388 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 14:14:44.610519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610402 2581 kubelet.go:398] "Adding apiserver pod source" Jul 06 14:14:44.610519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610416 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 14:14:44.610835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610822 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:14:44.610907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.610844 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 14:14:44.613341 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.613320 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 14:14:44.615750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.615736 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 14:14:44.615800 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.615767 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 14:14:44.617236 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617224 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617240 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617246 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617252 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617260 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617266 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617274 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617279 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617286 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617292 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 14:14:44.617298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617300 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 14:14:44.617551 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.617306 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 14:14:44.618137 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.618127 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 14:14:44.618173 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.618139 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 14:14:44.621592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.621579 2581 server.go:1268] "Started kubelet" Jul 06 14:14:44.621750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.621688 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 14:14:44.626367 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.626344 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 14:14:44.626883 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.626434 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 14:14:44.627005 ip-10-0-131-65 systemd[1]: Started Kubernetes Kubelet. Jul 06 14:14:44.627292 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.627259 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 14:14:44.627498 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.627368 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 14:14:44.627498 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.627468 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-65.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 14:14:44.628359 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.628338 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 14:14:44.629128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.629112 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 06 14:14:44.632829 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.631760 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e823064e63 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.621545059 +0000 UTC m=+0.316385298,LastTimestamp:2026-07-06 14:14:44.621545059 +0000 UTC m=+0.316385298,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.633428 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.633412 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 14:14:44.638933 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.638917 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 14:14:44.639008 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.638916 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 14:14:44.639058 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.639032 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:44.639714 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639696 2581 factory.go:55] Registering systemd factory Jul 06 14:14:44.639714 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639713 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 14:14:44.639836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639728 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 14:14:44.639836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639699 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 14:14:44.639836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639717 2581 factory.go:223] Registration of the systemd container factory successfully Jul 06 14:14:44.639987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639857 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 06 14:14:44.639987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639867 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 06 14:14:44.639987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639973 2581 factory.go:153] Registering CRI-O factory Jul 06 14:14:44.639987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.639986 2581 factory.go:223] Registration of the crio container factory successfully Jul 06 14:14:44.640161 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.640040 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 14:14:44.640161 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.640063 2581 factory.go:103] Registering Raw factory Jul 06 14:14:44.640161 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.640079 2581 manager.go:1196] Started watching for new ooms in manager Jul 06 14:14:44.640667 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.640541 2581 manager.go:349] Starting recovery of all containers Jul 06 14:14:44.642814 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.642785 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 14:14:44.650863 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.650828 2581 manager.go:354] Recovery completed Jul 06 14:14:44.655457 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.655442 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.656742 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.656722 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-65.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 14:14:44.657398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657380 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.657465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657413 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.657465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657429 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.657985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657970 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 14:14:44.657985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657982 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 14:14:44.658103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.657999 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 14:14:44.658103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.658018 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.658764 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.658748 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.658807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.658779 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.658807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.658789 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.659292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.659281 2581 policy_none.go:49] "None policy: Start" Jul 06 14:14:44.659326 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.659297 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 14:14:44.659326 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.659310 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 14:14:44.660346 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.660336 2581 policy_none.go:47] "Start" Jul 06 14:14:44.671267 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.671072 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e825295d44 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.65739706 +0000 UTC m=+0.352237301,LastTimestamp:2026-07-06 14:14:44.65739706 +0000 UTC m=+0.352237301,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.674427 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.674357 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e82529b6fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.657420029 +0000 UTC m=+0.352260270,LastTimestamp:2026-07-06 14:14:44.657420029 +0000 UTC m=+0.352260270,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.687531 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.687468 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e82529f4e5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.657435877 +0000 UTC m=+0.352276115,LastTimestamp:2026-07-06 14:14:44.657435877 +0000 UTC m=+0.352276115,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.692797 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.692783 2581 manager.go:337] "Starting Device Plugin manager" Jul 06 14:14:44.692871 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.692823 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 14:14:44.692871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.692834 2581 server.go:87] "Starting device plugin registration server" Jul 06 14:14:44.693122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.693110 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 14:14:44.693158 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.693124 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 14:14:44.693387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.693242 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 14:14:44.693387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.693329 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 14:14:44.693387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.693339 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 14:14:44.693898 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.693876 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 14:14:44.693970 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.693925 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:44.704823 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.704755 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-65.ec2.internal.18bfb8e825295d44\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e825295d44 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.65739706 +0000 UTC m=+0.352237301,LastTimestamp:2026-07-06 14:14:44.658764318 +0000 UTC m=+0.353604556,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.717578 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.717508 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-65.ec2.internal.18bfb8e82529b6fd\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e82529b6fd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.657420029 +0000 UTC m=+0.352260270,LastTimestamp:2026-07-06 14:14:44.658784049 +0000 UTC m=+0.353624287,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.723140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.723121 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z759g" Jul 06 14:14:44.740621 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.740548 2581 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-131-65.ec2.internal.18bfb8e82529f4e5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-65.ec2.internal.18bfb8e82529f4e5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-65.ec2.internal,UID:ip-10-0-131-65.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-131-65.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-131-65.ec2.internal,},FirstTimestamp:2026-07-06 14:14:44.657435877 +0000 UTC m=+0.352276115,LastTimestamp:2026-07-06 14:14:44.658793286 +0000 UTC m=+0.353633523,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-65.ec2.internal,}" Jul 06 14:14:44.742965 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.742948 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z759g" Jul 06 14:14:44.794350 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.794264 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.795385 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.795369 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.795474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.795399 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.795474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.795410 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.795474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.795436 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.806456 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.806425 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 14:14:44.807748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.807723 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 14:14:44.807849 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.807757 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 14:14:44.807849 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.807775 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 14:14:44.807849 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.807804 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 14:14:44.812438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.812421 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:14:44.815881 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.815867 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.815928 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.815888 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-65.ec2.internal\": node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:44.908860 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.908826 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal"] Jul 06 14:14:44.909041 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.908906 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.910712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.910692 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.910800 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.910730 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.910800 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.910742 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.911975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.911954 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.912109 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912082 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.912145 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912127 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.912729 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912716 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.912783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912742 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.912783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912755 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.912838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912719 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.912838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912816 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.912838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.912830 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.913866 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.913847 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.913967 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.913874 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:44.914505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.914485 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:44.914597 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.914513 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:44.914597 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.914523 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:44.937825 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.937802 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-65.ec2.internal\" not found" node="ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.942208 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.942169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/514e6a5df363bafe96e75142e83be5d6-config\") pod \"kube-apiserver-proxy-ip-10-0-131-65.ec2.internal\" (UID: \"514e6a5df363bafe96e75142e83be5d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.942291 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.942225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.942291 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.942246 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-65.ec2.internal\" not found" node="ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.942291 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:44.942251 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:44.971633 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:44.971606 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.042485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.042485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042488 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/514e6a5df363bafe96e75142e83be5d6-config\") pod \"kube-apiserver-proxy-ip-10-0-131-65.ec2.internal\" (UID: \"514e6a5df363bafe96e75142e83be5d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.042649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.042649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042554 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/514e6a5df363bafe96e75142e83be5d6-config\") pod \"kube-apiserver-proxy-ip-10-0-131-65.ec2.internal\" (UID: \"514e6a5df363bafe96e75142e83be5d6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.042649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042564 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.042649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.042593 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2418f30d7846ee11077261fb2d7cf20c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal\" (UID: \"2418f30d7846ee11077261fb2d7cf20c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.071978 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.071905 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.172831 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.172796 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.240940 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.240912 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.245224 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.245030 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:45.273886 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.273849 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.374423 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.374329 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.474758 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.474717 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.564275 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.564249 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 14:14:45.564671 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.564394 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 14:14:45.575559 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.575534 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.643066 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.643001 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:14:45.658475 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.658454 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:14:45.676327 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.676303 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.744401 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.744363 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 14:09:45 +0000 UTC" deadline="2027-12-29 05:14:28.047854511 +0000 UTC" Jul 06 14:14:45.744401 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.744397 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12974h59m42.303460877s" Jul 06 14:14:45.776683 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.776648 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.808053 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:45.808016 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514e6a5df363bafe96e75142e83be5d6.slice/crio-18b00d0114e0b241018b95a719f4a3615aa235da51d02b7ed9847922d6ad39fe WatchSource:0}: Error finding container 18b00d0114e0b241018b95a719f4a3615aa235da51d02b7ed9847922d6ad39fe: Status 404 returned error can't find the container with id 18b00d0114e0b241018b95a719f4a3615aa235da51d02b7ed9847922d6ad39fe Jul 06 14:14:45.812145 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.812127 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:14:45.812237 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:45.812186 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2418f30d7846ee11077261fb2d7cf20c.slice/crio-70eb4f6aba58221ae264a6434b05dc1dc2748e2e3dc79bc06bb80f4a9869ebc4 WatchSource:0}: Error finding container 70eb4f6aba58221ae264a6434b05dc1dc2748e2e3dc79bc06bb80f4a9869ebc4: Status 404 returned error can't find the container with id 70eb4f6aba58221ae264a6434b05dc1dc2748e2e3dc79bc06bb80f4a9869ebc4 Jul 06 14:14:45.846879 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.846854 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 14:14:45.847918 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.847899 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientMemory" Jul 06 14:14:45.848017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.847929 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 14:14:45.848017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:45.847939 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeHasSufficientPID" Jul 06 14:14:45.877336 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.877315 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:45.977785 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:45.977718 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:46.078423 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.078390 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-65.ec2.internal\" not found" Jul 06 14:14:46.146628 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.146603 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:14:46.239220 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.239116 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" Jul 06 14:14:46.265872 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.265850 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:14:46.266718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.266704 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" Jul 06 14:14:46.279081 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.279059 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 14:14:46.611673 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.611579 2581 apiserver.go:52] "Watching apiserver" Jul 06 14:14:46.622883 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.622856 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 14:14:46.624396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.623940 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mjjwx","kube-system/konnectivity-agent-n8wlh","openshift-cluster-node-tuning-operator/tuned-24pg9","openshift-image-registry/node-ca-sqrlm","openshift-network-operator/iptables-alerter-frchr","openshift-ovn-kubernetes/ovnkube-node-l49l9","kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft","openshift-dns/node-resolver-sqwhz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal","openshift-multus/multus-additional-cni-plugins-ccldf","openshift-multus/multus-r6rr7","openshift-multus/network-metrics-daemon-rkfvm","openshift-network-diagnostics/network-check-target-kx8z5"] Jul 06 14:14:46.625702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.625683 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.625829 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.625774 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:46.626890 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.626869 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.628093 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.628072 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.629371 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.629345 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.630874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.630856 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.632045 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.632029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.633481 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.633457 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.633611 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.633587 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:46.634143 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634122 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.634242 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634218 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.634327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634123 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.634327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634129 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.634460 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634441 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 14:14:46.634871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.634857 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.635423 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.635404 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p7jl8\"" Jul 06 14:14:46.636165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.636682 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636660 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-659kj\"" Jul 06 14:14:46.636813 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636712 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.636813 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636751 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 14:14:46.636813 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.637077 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636890 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 14:14:46.637077 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.636666 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 14:14:46.637077 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.637015 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 14:14:46.637352 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.637256 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6drsq\"" Jul 06 14:14:46.637749 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.637728 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:46.637837 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.637800 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:46.638558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.638537 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 14:14:46.638733 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.638718 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.639057 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.639037 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hknmx\"" Jul 06 14:14:46.639846 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.639575 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 14:14:46.639846 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.639838 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.640002 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.639900 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.641953 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.641931 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7cmzj\"" Jul 06 14:14:46.642063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.642045 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 14:14:46.642126 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.642104 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.642524 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.642506 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.643241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.643220 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 14:14:46.644176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.644159 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 14:14:46.644582 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.644562 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.644821 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.644804 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 14:14:46.645116 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.645099 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.645678 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.645658 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7fbp\"" Jul 06 14:14:46.645766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.645712 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xnkcl\"" Jul 06 14:14:46.645828 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.645812 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.650668 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysconfig\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.650766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cnibin\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.650766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650710 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/24b9bb35-2adc-4dc6-9d94-e505101d7c80-kube-api-access-sv9z8\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.650766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650737 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-bin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.650766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-host\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650785 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-systemd-units\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650835 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-netns\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cni-binary-copy\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650945 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-conf-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.650969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.650964 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e036f9e-256a-4a3c-a7c4-d2054a543008-agent-certs\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-node-log\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651075 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-registration-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4b8620-7304-4e70-9f89-434c533ae3f0-serviceca\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651127 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f2d0b9b-9c76-467e-9241-e41f2f29e260-iptables-alerter-script\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-kubelet-config\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651200 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kgqw\" (UniqueName: \"kubernetes.io/projected/849c6814-f736-4d4c-9ca7-f8433d2b5145-kube-api-access-8kgqw\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.651308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651275 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f2d0b9b-9c76-467e-9241-e41f2f29e260-host-slash\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-os-release\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-netns\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651469 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lhnp\" (UniqueName: \"kubernetes.io/projected/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-kube-api-access-8lhnp\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e036f9e-256a-4a3c-a7c4-d2054a543008-konnectivity-ca\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-bin\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651571 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-lib-modules\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-slash\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-systemd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdpnd\" (UniqueName: \"kubernetes.io/projected/953017ce-7e42-4305-bad7-c17ad54b7a2a-kube-api-access-xdpnd\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.651662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651669 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-kubernetes\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-run\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-kubelet\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-etc-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cnibin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmfxs\" (UniqueName: \"kubernetes.io/projected/e368bd33-196a-4aff-a96a-6922f3b7444d-kube-api-access-zmfxs\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651820 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-system-cni-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651839 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-log-socket\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58nqn\" (UniqueName: \"kubernetes.io/projected/135e0dca-d92d-483c-9e81-93ff18d532f4-kube-api-access-58nqn\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vb82\" (UniqueName: \"kubernetes.io/projected/2f2d0b9b-9c76-467e-9241-e41f2f29e260-kube-api-access-2vb82\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651927 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-systemd\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651941 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-tuned\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-sys-fs\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.652219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.651994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jl7t\" (UniqueName: \"kubernetes.io/projected/4e4b8620-7304-4e70-9f89-434c533ae3f0-kube-api-access-6jl7t\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-socket-dir-parent\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-kubelet\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652047 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-hostroot\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-system-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652084 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-device-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652105 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/135e0dca-d92d-483c-9e81-93ff18d532f4-hosts-file\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-modprobe-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-script-lib\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652210 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfs55\" (UniqueName: \"kubernetes.io/projected/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-kube-api-access-wfs55\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652281 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-ovn\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-netd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-env-overrides\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652338 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-multus\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-etc-kubernetes\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovn-node-metrics-cert\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.652897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/135e0dca-d92d-483c-9e81-93ff18d532f4-tmp-dir\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652482 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-k8s-cni-cncf-io\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-daemon-config\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652564 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-multus-certs\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652579 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-var-lib-kubelet\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652602 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-dbus\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-os-release\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652651 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652678 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-conf\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652726 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-sys\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-tmp\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-var-lib-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652767 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-config\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.653583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652780 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.654290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652799 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-socket-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.654290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.652813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4b8620-7304-4e70-9f89-434c533ae3f0-host\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.665089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.665066 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 14:14:46.665287 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.665263 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 14:14:46.665429 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.665169 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 14:14:46.665482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.665421 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 14:14:46.665558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.665538 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qwnnc\"" Jul 06 14:14:46.666683 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.666664 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 14:14:46.666781 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.666764 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 14:14:46.666964 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.666943 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xw66d\"" Jul 06 14:14:46.668971 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.668836 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 14:14:46.687947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.687920 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 14:14:46.753886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.753850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-system-cni-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.753892 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-log-socket\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.753921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58nqn\" (UniqueName: \"kubernetes.io/projected/135e0dca-d92d-483c-9e81-93ff18d532f4-kube-api-access-58nqn\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.753949 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vb82\" (UniqueName: \"kubernetes.io/projected/2f2d0b9b-9c76-467e-9241-e41f2f29e260-kube-api-access-2vb82\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.753971 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754001 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-systemd\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-tuned\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.754083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754070 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-sys-fs\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jl7t\" (UniqueName: \"kubernetes.io/projected/4e4b8620-7304-4e70-9f89-434c533ae3f0-kube-api-access-6jl7t\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-socket-dir-parent\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-kubelet\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-hostroot\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-system-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-device-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/135e0dca-d92d-483c-9e81-93ff18d532f4-hosts-file\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-modprobe-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754387 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-log-socket\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754386 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-script-lib\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfs55\" (UniqueName: \"kubernetes.io/projected/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-kube-api-access-wfs55\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-ovn\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-netd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754501 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-env-overrides\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754518 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-multus\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.754542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-etc-kubernetes\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754561 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovn-node-metrics-cert\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754577 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/135e0dca-d92d-483c-9e81-93ff18d532f4-tmp-dir\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-k8s-cni-cncf-io\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-daemon-config\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-multus-certs\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754697 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-var-lib-kubelet\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754739 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-dbus\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-os-release\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754814 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-conf\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-sys\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754844 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-tmp\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-var-lib-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.755412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754903 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-config\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-socket-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4b8620-7304-4e70-9f89-434c533ae3f0-host\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysconfig\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cnibin\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/24b9bb35-2adc-4dc6-9d94-e505101d7c80-kube-api-access-sv9z8\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-bin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-host\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755027 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-systemd-units\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755072 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-netns\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755090 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cni-binary-copy\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-conf-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e036f9e-256a-4a3c-a7c4-d2054a543008-agent-certs\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-node-log\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.756206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755210 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-registration-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755214 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-script-lib\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755234 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4b8620-7304-4e70-9f89-434c533ae3f0-serviceca\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755291 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f2d0b9b-9c76-467e-9241-e41f2f29e260-iptables-alerter-script\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-kubelet-config\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kgqw\" (UniqueName: \"kubernetes.io/projected/849c6814-f736-4d4c-9ca7-f8433d2b5145-kube-api-access-8kgqw\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f2d0b9b-9c76-467e-9241-e41f2f29e260-host-slash\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755403 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755413 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-sys\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755462 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-os-release\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-netns\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755587 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lhnp\" (UniqueName: \"kubernetes.io/projected/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-kube-api-access-8lhnp\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e036f9e-256a-4a3c-a7c4-d2054a543008-konnectivity-ca\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-bin\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755706 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-lib-modules\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.757165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-slash\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.755802 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-slash\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.756565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.756818 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-var-lib-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.756853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-sys-fs\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.756901 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-ovn\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.756938 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-netd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757242 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-env-overrides\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757329 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-multus\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757384 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-etc-kubernetes\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovnkube-config\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757482 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-socket-dir-parent\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757535 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-kubelet\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757548 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysconfig\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757572 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e4b8620-7304-4e70-9f89-434c533ae3f0-host\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757578 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cnibin\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-var-lib-cni-bin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-hostroot\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-systemd-units\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757676 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-netns\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757670 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-host\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-run-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757753 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-system-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757808 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-device-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757854 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-cni-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757862 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/135e0dca-d92d-483c-9e81-93ff18d532f4-hosts-file\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.757948 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-modprobe-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754331 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-systemd\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758022 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.754362 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-system-cni-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758186 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-socket-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cni-binary-copy\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758593 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/135e0dca-d92d-483c-9e81-93ff18d532f4-tmp-dir\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758757 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-conf-dir\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.758897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758844 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.759789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.758915 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-k8s-cni-cncf-io\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.759789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759526 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-multus-daemon-config\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.759789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-multus-certs\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.759789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759688 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-var-lib-kubelet\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.759789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759744 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.760035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759849 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-dbus\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.760035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759870 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-tuned\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.760035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.759929 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/24b9bb35-2adc-4dc6-9d94-e505101d7c80-os-release\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.760180 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.760085 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:46.760180 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760165 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kmqb4" Jul 06 14:14:46.760300 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.760185 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:47.260138956 +0000 UTC m=+2.954979185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760441 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-registration-dir\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-d\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-node-log\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760645 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-systemd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760709 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdpnd\" (UniqueName: \"kubernetes.io/projected/953017ce-7e42-4305-bad7-c17ad54b7a2a-kube-api-access-xdpnd\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760734 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/daf0384f-5fef-48ab-a5e9-219455c47099-kubelet-config\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-kubernetes\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e4b8620-7304-4e70-9f89-434c533ae3f0-serviceca\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-run\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760629 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-sysctl-conf\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-run-systemd\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760927 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f2d0b9b-9c76-467e-9241-e41f2f29e260-iptables-alerter-script\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760959 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-kubelet\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761026 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/953017ce-7e42-4305-bad7-c17ad54b7a2a-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.761844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.760900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-kubelet\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761070 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-etc-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cnibin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmfxs\" (UniqueName: \"kubernetes.io/projected/e368bd33-196a-4aff-a96a-6922f3b7444d-kube-api-access-zmfxs\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761240 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-host-run-netns\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/24b9bb35-2adc-4dc6-9d94-e505101d7c80-cni-binary-copy\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761305 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-host-cni-bin\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-etc-kubernetes\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761387 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-cnibin\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761426 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/849c6814-f736-4d4c-9ca7-f8433d2b5145-etc-openvswitch\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761440 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f2d0b9b-9c76-467e-9241-e41f2f29e260-host-slash\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.761554 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761578 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-lib-modules\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761557 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e368bd33-196a-4aff-a96a-6922f3b7444d-run\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-os-release\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761682 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e368bd33-196a-4aff-a96a-6922f3b7444d-tmp\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761554 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8e036f9e-256a-4a3c-a7c4-d2054a543008-konnectivity-ca\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.761787 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:47.261765225 +0000 UTC m=+2.956605457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:46.762912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.761920 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8e036f9e-256a-4a3c-a7c4-d2054a543008-agent-certs\") pod \"konnectivity-agent-n8wlh\" (UID: \"8e036f9e-256a-4a3c-a7c4-d2054a543008\") " pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.763804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.762379 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/849c6814-f736-4d4c-9ca7-f8433d2b5145-ovn-node-metrics-cert\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.775469 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.774073 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:14:46.775469 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.774104 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:14:46.775469 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.774117 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:46.775469 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:46.774234 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:47.274212496 +0000 UTC m=+2.969052743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:46.776170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.776133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9z8\" (UniqueName: \"kubernetes.io/projected/24b9bb35-2adc-4dc6-9d94-e505101d7c80-kube-api-access-sv9z8\") pod \"multus-additional-cni-plugins-ccldf\" (UID: \"24b9bb35-2adc-4dc6-9d94-e505101d7c80\") " pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:46.779534 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.779478 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfs55\" (UniqueName: \"kubernetes.io/projected/7093f96a-d9b5-4b0a-bd56-7bcce8632c0a-kube-api-access-wfs55\") pod \"multus-r6rr7\" (UID: \"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a\") " pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.779534 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.779515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kgqw\" (UniqueName: \"kubernetes.io/projected/849c6814-f736-4d4c-9ca7-f8433d2b5145-kube-api-access-8kgqw\") pod \"ovnkube-node-l49l9\" (UID: \"849c6814-f736-4d4c-9ca7-f8433d2b5145\") " pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.782034 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.782010 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lhnp\" (UniqueName: \"kubernetes.io/projected/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-kube-api-access-8lhnp\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:46.782222 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.782190 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vb82\" (UniqueName: \"kubernetes.io/projected/2f2d0b9b-9c76-467e-9241-e41f2f29e260-kube-api-access-2vb82\") pod \"iptables-alerter-frchr\" (UID: \"2f2d0b9b-9c76-467e-9241-e41f2f29e260\") " pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.787698 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.787666 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdpnd\" (UniqueName: \"kubernetes.io/projected/953017ce-7e42-4305-bad7-c17ad54b7a2a-kube-api-access-xdpnd\") pod \"aws-ebs-csi-driver-node-j9zft\" (UID: \"953017ce-7e42-4305-bad7-c17ad54b7a2a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.797797 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.797773 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kmqb4" Jul 06 14:14:46.804619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.804596 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmfxs\" (UniqueName: \"kubernetes.io/projected/e368bd33-196a-4aff-a96a-6922f3b7444d-kube-api-access-zmfxs\") pod \"tuned-24pg9\" (UID: \"e368bd33-196a-4aff-a96a-6922f3b7444d\") " pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.810343 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.808046 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58nqn\" (UniqueName: \"kubernetes.io/projected/135e0dca-d92d-483c-9e81-93ff18d532f4-kube-api-access-58nqn\") pod \"node-resolver-sqwhz\" (UID: \"135e0dca-d92d-483c-9e81-93ff18d532f4\") " pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:46.821532 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.820610 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jl7t\" (UniqueName: \"kubernetes.io/projected/4e4b8620-7304-4e70-9f89-434c533ae3f0-kube-api-access-6jl7t\") pod \"node-ca-sqrlm\" (UID: \"4e4b8620-7304-4e70-9f89-434c533ae3f0\") " pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.822121 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.822049 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" event={"ID":"2418f30d7846ee11077261fb2d7cf20c","Type":"ContainerStarted","Data":"70eb4f6aba58221ae264a6434b05dc1dc2748e2e3dc79bc06bb80f4a9869ebc4"} Jul 06 14:14:46.834912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.834846 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" event={"ID":"514e6a5df363bafe96e75142e83be5d6","Type":"ContainerStarted","Data":"18b00d0114e0b241018b95a719f4a3615aa235da51d02b7ed9847922d6ad39fe"} Jul 06 14:14:46.943081 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.942993 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:46.952748 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:46.952702 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e036f9e_256a_4a3c_a7c4_d2054a543008.slice/crio-cbbb73d32bf33ee9f670608acb4563e7100b590cfed3c55f4810c58d259ec97c WatchSource:0}: Error finding container cbbb73d32bf33ee9f670608acb4563e7100b590cfed3c55f4810c58d259ec97c: Status 404 returned error can't find the container with id cbbb73d32bf33ee9f670608acb4563e7100b590cfed3c55f4810c58d259ec97c Jul 06 14:14:46.953572 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.953549 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-24pg9" Jul 06 14:14:46.962139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.962111 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-r6rr7" Jul 06 14:14:46.967665 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.967511 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sqrlm" Jul 06 14:14:46.975719 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.975694 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frchr" Jul 06 14:14:46.983527 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.983505 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:14:46.984029 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:46.984002 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4b8620_7304_4e70_9f89_434c533ae3f0.slice/crio-01376f973d745ae851486b92baa0eb13e2fc39dd2949f79d6fa41483a3537038 WatchSource:0}: Error finding container 01376f973d745ae851486b92baa0eb13e2fc39dd2949f79d6fa41483a3537038: Status 404 returned error can't find the container with id 01376f973d745ae851486b92baa0eb13e2fc39dd2949f79d6fa41483a3537038 Jul 06 14:14:46.992096 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.992066 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" Jul 06 14:14:46.997822 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:46.997788 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849c6814_f736_4d4c_9ca7_f8433d2b5145.slice/crio-025f06abc52dd0b2fe28f92f42c92c1cdc8b6f44f10b6530f036786fac2d6845 WatchSource:0}: Error finding container 025f06abc52dd0b2fe28f92f42c92c1cdc8b6f44f10b6530f036786fac2d6845: Status 404 returned error can't find the container with id 025f06abc52dd0b2fe28f92f42c92c1cdc8b6f44f10b6530f036786fac2d6845 Jul 06 14:14:46.999047 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:46.998882 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sqwhz" Jul 06 14:14:47.004438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.004418 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ccldf" Jul 06 14:14:47.006460 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:47.006429 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953017ce_7e42_4305_bad7_c17ad54b7a2a.slice/crio-b12c25275381f75368faa1a1ef4d970a6ea6e9d411f0968ffaaa8be3fd9f5630 WatchSource:0}: Error finding container b12c25275381f75368faa1a1ef4d970a6ea6e9d411f0968ffaaa8be3fd9f5630: Status 404 returned error can't find the container with id b12c25275381f75368faa1a1ef4d970a6ea6e9d411f0968ffaaa8be3fd9f5630 Jul 06 14:14:47.013165 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:47.013139 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod135e0dca_d92d_483c_9e81_93ff18d532f4.slice/crio-3f53293ba8f677591cc079f40a4adb347463f95e1596b929a896b5c8a3788174 WatchSource:0}: Error finding container 3f53293ba8f677591cc079f40a4adb347463f95e1596b929a896b5c8a3788174: Status 404 returned error can't find the container with id 3f53293ba8f677591cc079f40a4adb347463f95e1596b929a896b5c8a3788174 Jul 06 14:14:47.018783 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:14:47.018751 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b9bb35_2adc_4dc6_9d94_e505101d7c80.slice/crio-96aa2cad7349a5658c888efccf3e6c6c187a69ad1b80139f78b81a3a6176e03f WatchSource:0}: Error finding container 96aa2cad7349a5658c888efccf3e6c6c187a69ad1b80139f78b81a3a6176e03f: Status 404 returned error can't find the container with id 96aa2cad7349a5658c888efccf3e6c6c187a69ad1b80139f78b81a3a6176e03f Jul 06 14:14:47.264272 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.264168 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:47.264443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.264286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:47.264443 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.264358 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:47.264568 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.264450 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:48.264427167 +0000 UTC m=+3.959267409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:47.264568 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.264451 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:47.264568 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.264517 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:48.264499255 +0000 UTC m=+3.959339500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:47.365649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.365612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:47.365838 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.365816 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:14:47.365903 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.365846 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:14:47.365903 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.365862 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:47.365992 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.365942 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:48.365914159 +0000 UTC m=+4.060754385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:47.801757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.801710 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:09:47 +0000 UTC" deadline="2028-03-20 05:47:21.335187682 +0000 UTC" Jul 06 14:14:47.801757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.801754 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14943h32m33.533438621s" Jul 06 14:14:47.809136 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.809115 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:47.809307 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:47.809259 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:47.838102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.838060 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerStarted","Data":"96aa2cad7349a5658c888efccf3e6c6c187a69ad1b80139f78b81a3a6176e03f"} Jul 06 14:14:47.839914 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.839881 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqwhz" event={"ID":"135e0dca-d92d-483c-9e81-93ff18d532f4","Type":"ContainerStarted","Data":"3f53293ba8f677591cc079f40a4adb347463f95e1596b929a896b5c8a3788174"} Jul 06 14:14:47.841301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.841265 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" event={"ID":"953017ce-7e42-4305-bad7-c17ad54b7a2a","Type":"ContainerStarted","Data":"b12c25275381f75368faa1a1ef4d970a6ea6e9d411f0968ffaaa8be3fd9f5630"} Jul 06 14:14:47.842933 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.842903 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"025f06abc52dd0b2fe28f92f42c92c1cdc8b6f44f10b6530f036786fac2d6845"} Jul 06 14:14:47.844229 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.844197 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frchr" event={"ID":"2f2d0b9b-9c76-467e-9241-e41f2f29e260","Type":"ContainerStarted","Data":"72695fc80904f52d84cb61ed53c307660a5eea4b3d48089d059001f33adc0ac8"} Jul 06 14:14:47.846975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.846916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sqrlm" event={"ID":"4e4b8620-7304-4e70-9f89-434c533ae3f0","Type":"ContainerStarted","Data":"01376f973d745ae851486b92baa0eb13e2fc39dd2949f79d6fa41483a3537038"} Jul 06 14:14:47.848379 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.848353 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r6rr7" event={"ID":"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a","Type":"ContainerStarted","Data":"14b555a56bf6250a803ce64881d2d3362fc35929b5da7ad2b614d793eaff1c8c"} Jul 06 14:14:47.849835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.849805 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24pg9" event={"ID":"e368bd33-196a-4aff-a96a-6922f3b7444d","Type":"ContainerStarted","Data":"01018617fdc50e2b88ce2f5f4180130230c3700fc4a4623ca34b9b9238fb52ab"} Jul 06 14:14:47.851460 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:47.851422 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n8wlh" event={"ID":"8e036f9e-256a-4a3c-a7c4-d2054a543008","Type":"ContainerStarted","Data":"cbbb73d32bf33ee9f670608acb4563e7100b590cfed3c55f4810c58d259ec97c"} Jul 06 14:14:48.273679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.273466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:48.273679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.273566 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:48.273985 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.273714 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:48.273985 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.273780 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:50.273760846 +0000 UTC m=+5.968601077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:48.274282 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.274209 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:48.274391 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.274286 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:50.27425165 +0000 UTC m=+5.969091879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:48.374684 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.374452 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:48.374800 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.374627 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:14:48.374800 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.374736 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:14:48.374800 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.374754 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:48.374941 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.374815 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:50.374796297 +0000 UTC m=+6.069636524 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:48.802967 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.802885 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 14:09:47 +0000 UTC" deadline="2028-04-17 05:30:43.310215632 +0000 UTC" Jul 06 14:14:48.802967 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.802942 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15615h15m54.507277623s" Jul 06 14:14:48.811537 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.810792 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:48.811537 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.810935 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:48.811537 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.811363 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:48.811537 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:48.811465 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:48.864167 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:48.863431 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" event={"ID":"514e6a5df363bafe96e75142e83be5d6","Type":"ContainerStarted","Data":"5b4754df8ea80ed5108c75b01953a3b7a6ba6fedf80fae880cbe76416c7dd59a"} Jul 06 14:14:49.808960 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:49.808924 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:49.809509 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:49.809072 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:49.881926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:49.881882 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" event={"ID":"2418f30d7846ee11077261fb2d7cf20c","Type":"ContainerStarted","Data":"c2856e32dea800e9284f4c9643185aff696af482474c7f706dddede0fcd033a6"} Jul 06 14:14:49.910588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:49.910521 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-65.ec2.internal" podStartSLOduration=3.910495377 podStartE2EDuration="3.910495377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:48.892814805 +0000 UTC m=+4.587655058" watchObservedRunningTime="2026-07-06 14:14:49.910495377 +0000 UTC m=+5.605335628" Jul 06 14:14:50.295328 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.295172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:50.295328 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.295245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:50.295568 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.295462 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:50.295568 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.295532 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:54.295511828 +0000 UTC m=+9.990352079 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:50.298684 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.295956 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:50.298684 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.296016 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:54.295997795 +0000 UTC m=+9.990838028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:50.396297 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.396098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:50.396297 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.396282 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:14:50.396484 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.396305 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:14:50.396484 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.396322 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:50.396484 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.396389 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:14:54.396369084 +0000 UTC m=+10.091209326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:50.809494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.809390 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:50.809982 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.809537 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:50.809982 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.809395 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:50.810104 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:50.810011 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:50.887412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.886487 2581 generic.go:356] "Generic (PLEG): container finished" podID="2418f30d7846ee11077261fb2d7cf20c" containerID="c2856e32dea800e9284f4c9643185aff696af482474c7f706dddede0fcd033a6" exitCode=0 Jul 06 14:14:50.887412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.886585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" event={"ID":"2418f30d7846ee11077261fb2d7cf20c","Type":"ContainerDied","Data":"c2856e32dea800e9284f4c9643185aff696af482474c7f706dddede0fcd033a6"} Jul 06 14:14:50.894513 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:50.894482 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-n8wlh" event={"ID":"8e036f9e-256a-4a3c-a7c4-d2054a543008","Type":"ContainerStarted","Data":"4075d98cd1fc95dbea1a4de0f8527bbb73aef6b1745ccf12de88c02f86da55b6"} Jul 06 14:14:51.809108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:51.808712 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:51.809108 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:51.808825 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:52.808397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:52.808362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:52.808793 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:52.808521 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:52.808793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:52.808584 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:52.808793 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:52.808714 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:53.809107 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:53.808569 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:53.809107 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:53.808702 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:54.327824 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:54.327792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:54.327987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:54.327852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:54.328046 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.327983 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:54.328046 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.328015 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:54.328137 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.328063 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:02.328039671 +0000 UTC m=+18.022879912 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:14:54.328137 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.328083 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:02.328073393 +0000 UTC m=+18.022913620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:14:54.429176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:54.429136 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:54.429364 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.429331 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:14:54.429364 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.429351 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:14:54.429364 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.429362 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:54.429524 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.429428 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:02.429406727 +0000 UTC m=+18.124246953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:14:54.808988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:54.808918 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:54.809120 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.809027 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:54.809734 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:54.809713 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:54.809849 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:54.809826 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:55.017717 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:55.017679 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:55.018273 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:55.018252 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:14:55.054482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:55.054428 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-n8wlh" podStartSLOduration=6.651272861 podStartE2EDuration="10.054400896s" totalImagesPullingTime="3.403128035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:50.962877061 +0000 UTC m=+6.657717315" watchObservedRunningTime="2026-07-06 14:14:55.054400896 +0000 UTC m=+10.749241145" Jul 06 14:14:55.808364 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:55.808331 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:55.808533 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:55.808461 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:56.808302 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:56.808264 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:56.808700 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:56.808281 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:56.808700 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:56.808423 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:56.808700 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:56.808516 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:57.808214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:57.808166 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:57.808438 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:57.808311 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:58.809317 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.809060 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:14:58.809750 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:58.809424 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:14:58.809750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.809133 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:14:58.809750 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:58.809539 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:14:58.914088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.913997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sqrlm" event={"ID":"4e4b8620-7304-4e70-9f89-434c533ae3f0","Type":"ContainerStarted","Data":"d3fc1f8cf5436bbe8b77f4d5fc8ca40101bf940e0f7182baf0b7487df20a1a9c"} Jul 06 14:14:58.917547 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.917400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" event={"ID":"2418f30d7846ee11077261fb2d7cf20c","Type":"ContainerStarted","Data":"3c617645f9cf357d407ef4edfa44bb94962c6f6bc6f8c0aeb32660cb53e6032a"} Jul 06 14:14:58.919151 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.919122 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-24pg9" event={"ID":"e368bd33-196a-4aff-a96a-6922f3b7444d","Type":"ContainerStarted","Data":"eadcec3458498ad67dfb0cd493e927df8eff791115612f5b64533193e1ad447a"} Jul 06 14:14:58.920958 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.920930 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerStarted","Data":"7d341974ded3b106a8241b021172f7a575f3ad939c8273a0330a054de0f75692"} Jul 06 14:14:58.923036 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.923007 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sqwhz" event={"ID":"135e0dca-d92d-483c-9e81-93ff18d532f4","Type":"ContainerStarted","Data":"03ce54a94e0cfd325279a7ed326268d67c3a152d991dbac547f94e9decbd647c"} Jul 06 14:14:58.925140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.925116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" event={"ID":"953017ce-7e42-4305-bad7-c17ad54b7a2a","Type":"ContainerStarted","Data":"c89750d15ffeaa142fa2d7879be765701151466ba14156bbb78edaa412411e1e"} Jul 06 14:14:58.965391 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:58.965340 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sqrlm" podStartSLOduration=2.646641457 podStartE2EDuration="13.965324345s" totalImagesPullingTime="11.318682888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:58.964961052 +0000 UTC m=+14.659801301" watchObservedRunningTime="2026-07-06 14:14:58.965324345 +0000 UTC m=+14.660164591" Jul 06 14:14:59.021609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.021545 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-24pg9" podStartSLOduration=2.608812487 podStartE2EDuration="14.021526189s" totalImagesPullingTime="11.412713702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:59.020982281 +0000 UTC m=+14.715822542" watchObservedRunningTime="2026-07-06 14:14:59.021526189 +0000 UTC m=+14.716366440" Jul 06 14:14:59.190217 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.190077 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-65.ec2.internal" podStartSLOduration=13.190057406 podStartE2EDuration="13.190057406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:59.189523125 +0000 UTC m=+14.884363374" watchObservedRunningTime="2026-07-06 14:14:59.190057406 +0000 UTC m=+14.884897653" Jul 06 14:14:59.250328 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.250272 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sqwhz" podStartSLOduration=2.955883343 podStartE2EDuration="14.250250626s" totalImagesPullingTime="11.294367283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:59.249969575 +0000 UTC m=+14.944809824" watchObservedRunningTime="2026-07-06 14:14:59.250250626 +0000 UTC m=+14.945090875" Jul 06 14:14:59.808631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.808597 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:14:59.808794 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:14:59.808717 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:14:59.928469 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.928406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frchr" event={"ID":"2f2d0b9b-9c76-467e-9241-e41f2f29e260","Type":"ContainerStarted","Data":"d300b7eee6aed536bde57e7b9491f3e4b340cd164842084f30f5ef8ec03a2771"} Jul 06 14:14:59.990843 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:14:59.990787 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-frchr" podStartSLOduration=3.662855264 podStartE2EDuration="14.990770219s" totalImagesPullingTime="11.327914955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:14:59.990559019 +0000 UTC m=+15.685399266" watchObservedRunningTime="2026-07-06 14:14:59.990770219 +0000 UTC m=+15.685610466" Jul 06 14:15:00.812140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:00.812109 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:00.812334 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:00.812254 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:00.812334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:00.812316 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:00.812468 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:00.812446 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:01.808976 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:01.808945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:01.809416 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:01.809057 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:02.397795 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.397759 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:02.397960 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.397813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:02.397960 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.397933 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:15:02.397960 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.397933 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:15:02.398083 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.397999 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:18.397980685 +0000 UTC m=+34.092820916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:15:02.398083 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.398016 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:18.398007745 +0000 UTC m=+34.092847985 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:15:02.498484 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.498446 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:02.498640 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.498610 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:15:02.498640 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.498631 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:15:02.498719 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.498643 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:15:02.498719 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.498709 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:18.498691303 +0000 UTC m=+34.193531531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:15:02.811357 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.811283 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:02.811985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.811292 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:02.811985 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.811387 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:02.811985 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:02.811487 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:02.937180 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.937132 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="7d341974ded3b106a8241b021172f7a575f3ad939c8273a0330a054de0f75692" exitCode=0 Jul 06 14:15:02.937180 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:02.937183 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"7d341974ded3b106a8241b021172f7a575f3ad939c8273a0330a054de0f75692"} Jul 06 14:15:03.808691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:03.808655 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:03.808866 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:03.808789 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:04.809680 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:04.809645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:04.810157 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:04.809752 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:04.810157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:04.809822 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:04.810157 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:04.809955 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:05.808792 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:05.808758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:05.808949 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:05.808893 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:06.808310 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:06.808284 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:06.808732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:06.808289 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:06.808732 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:06.808388 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:06.808732 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:06.808484 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:07.408916 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.408686 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 14:15:07.707341 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.706722 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T14:15:07.408899335Z","UUID":"af415bc8-2f97-4cb9-937e-b9df8f0d8049","Handler":null,"Name":"","Endpoint":""} Jul 06 14:15:07.708639 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.708617 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 14:15:07.708771 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.708658 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 14:15:07.808895 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.808862 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:07.809389 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:07.809003 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:07.948638 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.948542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" event={"ID":"953017ce-7e42-4305-bad7-c17ad54b7a2a","Type":"ContainerStarted","Data":"70e27d2f15f0226a1b73be3213451d39006701cc0d9734ca9829967a94fc71da"} Jul 06 14:15:07.952106 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952078 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"6e2d03b9adb97f71cdc1e2820e675ddcac678c9b5b29c367ff0338264de59a7d"} Jul 06 14:15:07.952253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952158 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"bcf661fda4f1ff64634f4ac92d08e7fb893ec0b46c52659e4c3e562418009764"} Jul 06 14:15:07.952253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952230 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"75eaa450f720d6cf9ae03395fbbc5d2be8391837b999ddc1dac482c83cfbc174"} Jul 06 14:15:07.952368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952278 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"691bcc878cb5d6a4657c5ed8ab41e2518517c2aba25d9bb3e6f7d7f4b760f33c"} Jul 06 14:15:07.952368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952319 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"f8df2ea4b86971a3576e55aa8deafec9a9d8b091d62f55288a310a12a9d6925e"} Jul 06 14:15:07.952368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.952356 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"70ab5df8cffbac19bf916e4d00a01afb20e49ee643c3b9412b29f9de988bab4c"} Jul 06 14:15:07.953917 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:07.953889 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-r6rr7" event={"ID":"7093f96a-d9b5-4b0a-bd56-7bcce8632c0a","Type":"ContainerStarted","Data":"ba9627a0b8b7a2b2fe18003652b084c855b590c63b68fb8ee10640061516a7c3"} Jul 06 14:15:08.808253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.808211 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:08.808253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.808240 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:08.808489 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:08.808365 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:08.808586 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:08.808547 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:08.932636 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.932595 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:15:08.933045 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.932724 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 14:15:08.933153 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.933137 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-n8wlh" Jul 06 14:15:08.957608 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.957568 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" event={"ID":"953017ce-7e42-4305-bad7-c17ad54b7a2a","Type":"ContainerStarted","Data":"4fd7b95f9221d470c95e2578de96353f3d86a9f323f63f3e88929a7dce5f921e"} Jul 06 14:15:08.976433 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:08.976366 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-r6rr7" podStartSLOduration=3.711404149 podStartE2EDuration="23.976353121s" totalImagesPullingTime="20.264948972s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:07.986862238 +0000 UTC m=+23.681702484" watchObservedRunningTime="2026-07-06 14:15:08.976353121 +0000 UTC m=+24.671193373" Jul 06 14:15:09.004676 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:09.004630 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9zft" podStartSLOduration=3.129867291 podStartE2EDuration="24.004616785s" totalImagesPullingTime="20.874749494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:09.004479318 +0000 UTC m=+24.699319566" watchObservedRunningTime="2026-07-06 14:15:09.004616785 +0000 UTC m=+24.699457031" Jul 06 14:15:09.807981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:09.807945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:09.808133 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:09.808046 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:09.962245 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:09.962170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"49290692971dcfe194b29919c5da311e6b5d43ac3aedbac465709669c3532e6e"} Jul 06 14:15:10.808023 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:10.807987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:10.808023 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:10.808013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:10.808308 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:10.808097 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:10.808308 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:10.808229 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:11.808188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:11.808159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:11.808718 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:11.808297 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:11.971441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:11.971408 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" event={"ID":"849c6814-f736-4d4c-9ca7-f8433d2b5145","Type":"ContainerStarted","Data":"6efa95b80d743b81ab8381e71c177888674d9e7cc49848c3c7ecce4eb37369f5"} Jul 06 14:15:11.971711 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:11.971693 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:11.971802 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:11.971763 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:11.986685 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:11.986661 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:12.007905 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:12.007848 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" podStartSLOduration=6.763769666 podStartE2EDuration="27.007829445s" totalImagesPullingTime="20.244059779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:12.007674759 +0000 UTC m=+27.702515007" watchObservedRunningTime="2026-07-06 14:15:12.007829445 +0000 UTC m=+27.702669693" Jul 06 14:15:12.808453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:12.808423 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:12.808938 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:12.808539 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:12.808938 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:12.808602 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:12.808938 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:12.808722 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:12.974237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:12.974207 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:12.989809 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:12.989786 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:13.808893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:13.808619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:13.809305 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:13.808897 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:14.132343 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.131875 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjjwx"] Jul 06 14:15:14.132343 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.132058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:14.132343 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:14.132157 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:14.132764 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.132730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kx8z5"] Jul 06 14:15:14.132862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.132818 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:14.132915 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:14.132892 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:14.134316 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.134285 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkfvm"] Jul 06 14:15:14.134405 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:14.134389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:14.134512 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:14.134493 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:15.807999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:15.807962 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:15.807999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:15.807989 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:15.808505 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:15.808128 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:15.808505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:15.808170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:15.808505 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:15.808281 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:15.808505 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:15.808356 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:15.983515 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:15.983482 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="8ea50039a829f568c8fbe1aa921700e7b214f1506df95f71a0585bc6b4923b55" exitCode=0 Jul 06 14:15:15.983647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:15.983533 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"8ea50039a829f568c8fbe1aa921700e7b214f1506df95f71a0585bc6b4923b55"} Jul 06 14:15:17.808403 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:17.808368 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:17.808403 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:17.808399 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:17.808736 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:17.808372 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:17.808736 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:17.808504 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:17.808736 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:17.808549 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:17.808736 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:17.808638 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:18.413930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:18.413696 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:18.413930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:18.413920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:18.414144 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.413847 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 14:15:18.414144 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.414000 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:15:18.414144 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.414012 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret podName:daf0384f-5fef-48ab-a5e9-219455c47099 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:50.413991995 +0000 UTC m=+66.108832225 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret") pod "global-pull-secret-syncer-mjjwx" (UID: "daf0384f-5fef-48ab-a5e9-219455c47099") : object "kube-system"/"original-pull-secret" not registered Jul 06 14:15:18.414144 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.414039 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:50.414026766 +0000 UTC m=+66.108866991 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 14:15:18.515130 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:18.515100 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:18.515384 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.515248 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 14:15:18.515384 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.515262 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 14:15:18.515384 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.515273 2581 projected.go:196] Error preparing data for projected volume kube-api-access-2slgj for pod openshift-network-diagnostics/network-check-target-kx8z5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:15:18.515384 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:18.515321 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj podName:aaf11e30-18ff-48e9-9116-bc8b56bdc1c8 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:50.515307885 +0000 UTC m=+66.210148110 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2slgj" (UniqueName: "kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj") pod "network-check-target-kx8z5" (UID: "aaf11e30-18ff-48e9-9116-bc8b56bdc1c8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 14:15:18.993871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:18.993835 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="b067f12e5caf6deb1811289b2a033e333c523b7ade57b8461ab7f66df899ee5d" exitCode=0 Jul 06 14:15:18.993871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:18.993873 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"b067f12e5caf6deb1811289b2a033e333c523b7ade57b8461ab7f66df899ee5d"} Jul 06 14:15:19.808534 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:19.808500 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:19.808688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:19.808502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:19.808688 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:19.808604 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kx8z5" podUID="aaf11e30-18ff-48e9-9116-bc8b56bdc1c8" Jul 06 14:15:19.808688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:19.808638 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:19.808793 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:19.808727 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mjjwx" podUID="daf0384f-5fef-48ab-a5e9-219455c47099" Jul 06 14:15:19.808827 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:19.808790 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkfvm" podUID="d4e36c94-f9aa-45b2-b1f6-90974c8dca06" Jul 06 14:15:20.114814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.114783 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-65.ec2.internal" event="NodeReady" Jul 06 14:15:20.115285 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.114940 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 14:15:20.174703 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.174678 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:15:20.196178 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.196152 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gtpmk"] Jul 06 14:15:20.196178 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.196176 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.201940 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.201914 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 14:15:20.202065 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.201973 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 14:15:20.202381 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.202247 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 14:15:20.202855 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.202839 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gqm6v\"" Jul 06 14:15:20.210452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.210296 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hz5n9"] Jul 06 14:15:20.210452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.210432 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.217723 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.217699 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 14:15:20.222606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.222582 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-txr87\"" Jul 06 14:15:20.222859 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.222830 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 14:15:20.225166 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.225146 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 14:15:20.226531 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.226508 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 14:15:20.228234 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.228213 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:15:20.228324 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.228252 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtpmk"] Jul 06 14:15:20.228375 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.228292 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.230939 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.230918 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hz5n9"] Jul 06 14:15:20.239143 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.239115 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 14:15:20.239261 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.239160 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7sgw\"" Jul 06 14:15:20.239261 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.239162 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 14:15:20.328861 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328786 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj4jd\" (UniqueName: \"kubernetes.io/projected/9769f741-41f9-406f-8361-4164f03fbd8a-kube-api-access-dj4jd\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.328861 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328835 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.328861 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328946 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.328997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9769f741-41f9-406f-8361-4164f03fbd8a-tmp-dir\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329064 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9769f741-41f9-406f-8361-4164f03fbd8a-config-volume\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5c2p\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/685b830e-dc3d-4669-9524-ee547345608a-cert\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.329425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.329425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.329176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bwn\" (UniqueName: \"kubernetes.io/projected/685b830e-dc3d-4669-9524-ee547345608a-kube-api-access-68bwn\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.429787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.429787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429809 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429875 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9769f741-41f9-406f-8361-4164f03fbd8a-tmp-dir\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429899 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9769f741-41f9-406f-8361-4164f03fbd8a-config-volume\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5c2p\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429965 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/685b830e-dc3d-4669-9524-ee547345608a-cert\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.430004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.429998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430045 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bwn\" (UniqueName: \"kubernetes.io/projected/685b830e-dc3d-4669-9524-ee547345608a-kube-api-access-68bwn\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.430058 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.430078 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.430085 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430075 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj4jd\" (UniqueName: \"kubernetes.io/projected/9769f741-41f9-406f-8361-4164f03fbd8a-kube-api-access-dj4jd\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.430153 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:20.930132143 +0000 UTC m=+36.624972386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430281 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9769f741-41f9-406f-8361-4164f03fbd8a-tmp-dir\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.430422 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.430295 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:20.930278429 +0000 UTC m=+36.625118655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:20.430874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430589 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.430874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.430608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9769f741-41f9-406f-8361-4164f03fbd8a-config-volume\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.431267 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.431248 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.434397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.434369 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.434479 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.434410 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.434479 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.434435 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/685b830e-dc3d-4669-9524-ee547345608a-cert\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.457654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.457626 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.457944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.457921 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5c2p\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.459077 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.459057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bwn\" (UniqueName: \"kubernetes.io/projected/685b830e-dc3d-4669-9524-ee547345608a-kube-api-access-68bwn\") pod \"ingress-canary-gtpmk\" (UID: \"685b830e-dc3d-4669-9524-ee547345608a\") " pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.459173 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.459154 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj4jd\" (UniqueName: \"kubernetes.io/projected/9769f741-41f9-406f-8361-4164f03fbd8a-kube-api-access-dj4jd\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.520237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.520210 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gtpmk" Jul 06 14:15:20.535428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.535395 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:20.700509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.700483 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gtpmk"] Jul 06 14:15:20.703664 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:15:20.703635 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685b830e_dc3d_4669_9524_ee547345608a.slice/crio-8de6699946d7b7a861e149d8374966d1764a34509ef2d58103b79f5904666753 WatchSource:0}: Error finding container 8de6699946d7b7a861e149d8374966d1764a34509ef2d58103b79f5904666753: Status 404 returned error can't find the container with id 8de6699946d7b7a861e149d8374966d1764a34509ef2d58103b79f5904666753 Jul 06 14:15:20.933857 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.933774 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:20.933857 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.933815 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:20.934039 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.933915 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:20.934039 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.933916 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:20.934039 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.933925 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:20.934039 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.933990 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:21.933964385 +0000 UTC m=+37.628804610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:20.934039 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:20.934007 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:21.93400058 +0000 UTC m=+37.628840804 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:20.999630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:20.999595 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtpmk" event={"ID":"685b830e-dc3d-4669-9524-ee547345608a","Type":"ContainerStarted","Data":"8de6699946d7b7a861e149d8374966d1764a34509ef2d58103b79f5904666753"} Jul 06 14:15:21.002111 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.002087 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="0851e20ebd5f46f96d2e1c2bfeada3264811963804a7a3f471582aab9fabf850" exitCode=0 Jul 06 14:15:21.002268 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.002118 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"0851e20ebd5f46f96d2e1c2bfeada3264811963804a7a3f471582aab9fabf850"} Jul 06 14:15:21.140712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.140527 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:21.351299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.351268 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:21.599459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.599417 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:21.809036 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.809003 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:21.809461 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.809442 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:21.809847 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.809830 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:21.813082 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.812892 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:15:21.815364 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.815132 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:15:21.815828 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.815797 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mlf5t\"" Jul 06 14:15:21.816058 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.816043 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:15:21.816625 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.816600 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:15:21.816956 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.816937 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5hdv8\"" Jul 06 14:15:21.941286 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.941257 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:21.941480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:21.941309 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:21.941480 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:21.941459 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:21.941480 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:21.941474 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:21.941647 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:21.941533 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:23.94151484 +0000 UTC m=+39.636355087 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:21.942229 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:21.942181 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:21.942380 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:21.942259 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:23.942245829 +0000 UTC m=+39.637086054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:22.293763 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:22.293725 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:22.493992 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:22.493961 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:23.295996 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:23.295958 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:23.444441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:23.444404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" limit=1024 Jul 06 14:15:23.962732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:23.962693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:23.962956 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:23.962756 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:23.962956 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:23.962862 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:23.962956 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:23.962902 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:23.962956 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:23.962916 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:23.962956 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:23.962941 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:27.962919892 +0000 UTC m=+43.657760120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:23.963243 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:23.962965 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:27.962949635 +0000 UTC m=+43.657789859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:25.013675 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:25.013642 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gtpmk" event={"ID":"685b830e-dc3d-4669-9524-ee547345608a","Type":"ContainerStarted","Data":"e8d1c1a67491babacd1e42c8e3d14681182099a15458147fd8cfacc22ee04e96"} Jul 06 14:15:25.054621 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:25.054568 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gtpmk" podStartSLOduration=1.440405679 podStartE2EDuration="5.054550564s" totalImagesPullingTime="3.614144885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:25.049621276 +0000 UTC m=+40.744461524" watchObservedRunningTime="2026-07-06 14:15:25.054550564 +0000 UTC m=+40.749390788" Jul 06 14:15:27.997880 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:27.997708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:27.997880 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:27.997863 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:27.998301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:27.997896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:27.998301 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:27.997925 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:35.997908681 +0000 UTC m=+51.692748906 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:27.998301 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:27.997970 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:27.998301 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:27.997980 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:27.998301 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:27.998022 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:35.99800914 +0000 UTC m=+51.692849366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:28.024119 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:28.024086 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="582dc04a087cc318b466a3a1acbd77abb74087e03279abb6b032bd63dc28140e" exitCode=0 Jul 06 14:15:28.024269 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:28.024149 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"582dc04a087cc318b466a3a1acbd77abb74087e03279abb6b032bd63dc28140e"} Jul 06 14:15:29.029165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:29.029136 2581 generic.go:356] "Generic (PLEG): container finished" podID="24b9bb35-2adc-4dc6-9d94-e505101d7c80" containerID="d0d55c09440e454e95d184386dc088fde214fa488f80ea5dc4b6f95fa18985a9" exitCode=0 Jul 06 14:15:29.029590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:29.029208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerDied","Data":"d0d55c09440e454e95d184386dc088fde214fa488f80ea5dc4b6f95fa18985a9"} Jul 06 14:15:30.035159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:30.035119 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ccldf" event={"ID":"24b9bb35-2adc-4dc6-9d94-e505101d7c80","Type":"ContainerStarted","Data":"61fb88257d8bada2fd94fb43645b72fa4c1ab6d19141b46b9a649adfd5f17352"} Jul 06 14:15:30.071904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:30.071852 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ccldf" podStartSLOduration=12.613240143 podStartE2EDuration="45.071837863s" totalImagesPullingTime="32.45859772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:30.069868069 +0000 UTC m=+45.764708316" watchObservedRunningTime="2026-07-06 14:15:30.071837863 +0000 UTC m=+45.766678110" Jul 06 14:15:36.058552 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:36.058518 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:36.058591 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:36.058664 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:36.058676 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:36.058678 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:36.058733 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:15:52.058720268 +0000 UTC m=+67.753560493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:36.058999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:36.058745 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:15:52.058739912 +0000 UTC m=+67.753580138 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:44.990337 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:44.990308 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l49l9" Jul 06 14:15:47.836738 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.836700 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bz6fs"] Jul 06 14:15:47.839587 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.839570 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:47.842818 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.842796 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-8474t\"" Jul 06 14:15:47.842942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.842861 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 14:15:47.844290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.844270 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:15:47.844419 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.844275 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 14:15:47.844419 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.844275 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 14:15:47.849357 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.849339 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 14:15:47.881294 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.881260 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bz6fs"] Jul 06 14:15:47.938148 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.938106 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-config\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:47.938148 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.938151 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-trusted-ca\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:47.938445 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.938253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f25865-ef65-43b0-849d-bf8786f97e50-serving-cert\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:47.938445 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:47.938284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmr7\" (UniqueName: \"kubernetes.io/projected/e8f25865-ef65-43b0-849d-bf8786f97e50-kube-api-access-8dmr7\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.039626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.039590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmr7\" (UniqueName: \"kubernetes.io/projected/e8f25865-ef65-43b0-849d-bf8786f97e50-kube-api-access-8dmr7\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.039802 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.039690 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-config\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.039802 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.039717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-trusted-ca\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.039802 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.039771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f25865-ef65-43b0-849d-bf8786f97e50-serving-cert\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.040420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.040396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-config\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.040584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.040566 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e8f25865-ef65-43b0-849d-bf8786f97e50-trusted-ca\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.047504 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.047483 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e8f25865-ef65-43b0-849d-bf8786f97e50-serving-cert\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.055082 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.055047 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmr7\" (UniqueName: \"kubernetes.io/projected/e8f25865-ef65-43b0-849d-bf8786f97e50-kube-api-access-8dmr7\") pod \"console-operator-68b757fb67-bz6fs\" (UID: \"e8f25865-ef65-43b0-849d-bf8786f97e50\") " pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.148259 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.148164 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:48.331673 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:48.331637 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-bz6fs"] Jul 06 14:15:48.343119 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:15:48.343089 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f25865_ef65_43b0_849d_bf8786f97e50.slice/crio-d7d38b63a2062e28ea9cf0823498e612d95cd37dc2fccdafdb50b3af4baa7cb4 WatchSource:0}: Error finding container d7d38b63a2062e28ea9cf0823498e612d95cd37dc2fccdafdb50b3af4baa7cb4: Status 404 returned error can't find the container with id d7d38b63a2062e28ea9cf0823498e612d95cd37dc2fccdafdb50b3af4baa7cb4 Jul 06 14:15:49.080745 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:49.080713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" event={"ID":"e8f25865-ef65-43b0-849d-bf8786f97e50","Type":"ContainerStarted","Data":"d7d38b63a2062e28ea9cf0823498e612d95cd37dc2fccdafdb50b3af4baa7cb4"} Jul 06 14:15:50.456448 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.456401 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:15:50.456930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.456458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:50.461812 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.461785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 14:15:50.461938 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.461794 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 14:15:50.466975 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:50.466955 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 14:15:50.467073 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:50.467029 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs podName:d4e36c94-f9aa-45b2-b1f6-90974c8dca06 nodeName:}" failed. No retries permitted until 2026-07-06 14:16:54.467006775 +0000 UTC m=+130.161847004 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs") pod "network-metrics-daemon-rkfvm" (UID: "d4e36c94-f9aa-45b2-b1f6-90974c8dca06") : secret "metrics-daemon-secret" not found Jul 06 14:15:50.469326 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.469303 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/daf0384f-5fef-48ab-a5e9-219455c47099-original-pull-secret\") pod \"global-pull-secret-syncer-mjjwx\" (UID: \"daf0384f-5fef-48ab-a5e9-219455c47099\") " pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:50.556887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.556849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:50.560634 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.560597 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 14:15:50.571042 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.571020 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 14:15:50.580982 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.580950 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2slgj\" (UniqueName: \"kubernetes.io/projected/aaf11e30-18ff-48e9-9116-bc8b56bdc1c8-kube-api-access-2slgj\") pod \"network-check-target-kx8z5\" (UID: \"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8\") " pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:50.635293 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.635255 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5hdv8\"" Jul 06 14:15:50.636154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.635918 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mjjwx" Jul 06 14:15:50.641595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.641564 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:50.996261 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:50.996231 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mjjwx"] Jul 06 14:15:51.000840 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:15:51.000811 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf0384f_5fef_48ab_a5e9_219455c47099.slice/crio-6bd6d1823c810613345f614d6a3c89593999cad0f3f27a0331051cedb48f5c4b WatchSource:0}: Error finding container 6bd6d1823c810613345f614d6a3c89593999cad0f3f27a0331051cedb48f5c4b: Status 404 returned error can't find the container with id 6bd6d1823c810613345f614d6a3c89593999cad0f3f27a0331051cedb48f5c4b Jul 06 14:15:51.025864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.025821 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kx8z5"] Jul 06 14:15:51.080046 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:15:51.080007 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf11e30_18ff_48e9_9116_bc8b56bdc1c8.slice/crio-687f98919eefe5f541a555e40c2686d3b8c2dfc1e9993e63e24660aead39affe WatchSource:0}: Error finding container 687f98919eefe5f541a555e40c2686d3b8c2dfc1e9993e63e24660aead39affe: Status 404 returned error can't find the container with id 687f98919eefe5f541a555e40c2686d3b8c2dfc1e9993e63e24660aead39affe Jul 06 14:15:51.086819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.086772 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/0.log" Jul 06 14:15:51.086922 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.086843 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8f25865-ef65-43b0-849d-bf8786f97e50" containerID="1ffb004f7de220d5d4d7d693874b0231f4ebc928808fb0c7669f665884cf9992" exitCode=255 Jul 06 14:15:51.086922 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.086905 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" event={"ID":"e8f25865-ef65-43b0-849d-bf8786f97e50","Type":"ContainerDied","Data":"1ffb004f7de220d5d4d7d693874b0231f4ebc928808fb0c7669f665884cf9992"} Jul 06 14:15:51.087166 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.087140 2581 scope.go:117] "RemoveContainer" containerID="1ffb004f7de220d5d4d7d693874b0231f4ebc928808fb0c7669f665884cf9992" Jul 06 14:15:51.088114 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.088094 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjjwx" event={"ID":"daf0384f-5fef-48ab-a5e9-219455c47099","Type":"ContainerStarted","Data":"6bd6d1823c810613345f614d6a3c89593999cad0f3f27a0331051cedb48f5c4b"} Jul 06 14:15:51.089092 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:51.089073 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kx8z5" event={"ID":"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8","Type":"ContainerStarted","Data":"687f98919eefe5f541a555e40c2686d3b8c2dfc1e9993e63e24660aead39affe"} Jul 06 14:15:52.068065 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.068026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.068088 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.068169 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.068208 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.068224 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74f77d4b8b-sdpbx: secret "image-registry-tls" not found Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.068250 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls podName:9769f741-41f9-406f-8361-4164f03fbd8a nodeName:}" failed. No retries permitted until 2026-07-06 14:16:24.068234293 +0000 UTC m=+99.763074517 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls") pod "dns-default-hz5n9" (UID: "9769f741-41f9-406f-8361-4164f03fbd8a") : secret "dns-default-metrics-tls" not found Jul 06 14:15:52.068612 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.068313 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls podName:fc11d03b-01e1-4b5e-bed2-ccce79b70b65 nodeName:}" failed. No retries permitted until 2026-07-06 14:16:24.068298795 +0000 UTC m=+99.763139042 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls") pod "image-registry-74f77d4b8b-sdpbx" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65") : secret "image-registry-tls" not found Jul 06 14:15:52.093756 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.093725 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/1.log" Jul 06 14:15:52.094344 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.094325 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/0.log" Jul 06 14:15:52.094453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.094363 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8f25865-ef65-43b0-849d-bf8786f97e50" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" exitCode=255 Jul 06 14:15:52.094453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.094431 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" event={"ID":"e8f25865-ef65-43b0-849d-bf8786f97e50","Type":"ContainerDied","Data":"b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff"} Jul 06 14:15:52.094551 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.094483 2581 scope.go:117] "RemoveContainer" containerID="1ffb004f7de220d5d4d7d693874b0231f4ebc928808fb0c7669f665884cf9992" Jul 06 14:15:52.094727 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:52.094706 2581 scope.go:117] "RemoveContainer" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" Jul 06 14:15:52.094928 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:52.094898 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bz6fs_openshift-console-operator(e8f25865-ef65-43b0-849d-bf8786f97e50)\"" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podUID="e8f25865-ef65-43b0-849d-bf8786f97e50" Jul 06 14:15:53.098882 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.098851 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/1.log" Jul 06 14:15:53.099349 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.099332 2581 scope.go:117] "RemoveContainer" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" Jul 06 14:15:53.099526 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:53.099496 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bz6fs_openshift-console-operator(e8f25865-ef65-43b0-849d-bf8786f97e50)\"" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podUID="e8f25865-ef65-43b0-849d-bf8786f97e50" Jul 06 14:15:53.699090 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.697885 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hcjlq"] Jul 06 14:15:53.700304 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.700278 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.707443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.707417 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 14:15:53.716218 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.716179 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 14:15:53.716463 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.716449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 14:15:53.716609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.716516 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lm5gg\"" Jul 06 14:15:53.716609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.716449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 14:15:53.755562 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.755371 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcjlq"] Jul 06 14:15:53.783625 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.783589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.783831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.783653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24889622-5265-4405-8061-b2cd283c091f-data-volume\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.783831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.783676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8j8\" (UniqueName: \"kubernetes.io/projected/24889622-5265-4405-8061-b2cd283c091f-kube-api-access-vw8j8\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.783831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.783814 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24889622-5265-4405-8061-b2cd283c091f-crio-socket\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.783979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.783853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24889622-5265-4405-8061-b2cd283c091f-data-volume\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884297 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8j8\" (UniqueName: \"kubernetes.io/projected/24889622-5265-4405-8061-b2cd283c091f-kube-api-access-vw8j8\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884369 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24889622-5265-4405-8061-b2cd283c091f-crio-socket\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884428 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/24889622-5265-4405-8061-b2cd283c091f-crio-socket\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884538 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884600 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884720 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.884637 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/24889622-5265-4405-8061-b2cd283c091f-data-volume\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:53.884773 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:53.884722 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:15:53.884773 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:53.884739 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:15:53.884850 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:53.884788 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:54.384771847 +0000 UTC m=+70.079612076 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : configmap "kube-rbac-proxy" not found Jul 06 14:15:53.884850 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:53.884803 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:54.384796468 +0000 UTC m=+70.079636693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:15:53.941920 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:53.941884 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8j8\" (UniqueName: \"kubernetes.io/projected/24889622-5265-4405-8061-b2cd283c091f-kube-api-access-vw8j8\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:54.388412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.388370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:54.388844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.388440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:54.388844 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:54.388498 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:15:54.388844 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:54.388564 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:55.388550174 +0000 UTC m=+71.083390399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : configmap "kube-rbac-proxy" not found Jul 06 14:15:54.388844 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:54.388596 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:15:54.388844 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:54.388656 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:55.38863796 +0000 UTC m=+71.083478189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:15:54.634413 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.634384 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z"] Jul 06 14:15:54.637451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.637430 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" Jul 06 14:15:54.641908 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.641845 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 14:15:54.644215 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.644174 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 14:15:54.645816 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.645793 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-wjrdd\"" Jul 06 14:15:54.661108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.661073 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z"] Jul 06 14:15:54.691328 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.691292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bwq5\" (UniqueName: \"kubernetes.io/projected/22ed4150-a056-4e6d-be0d-3233db98adac-kube-api-access-7bwq5\") pod \"migrator-d57d58cc9-2zl8z\" (UID: \"22ed4150-a056-4e6d-be0d-3233db98adac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" Jul 06 14:15:54.792299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.792258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bwq5\" (UniqueName: \"kubernetes.io/projected/22ed4150-a056-4e6d-be0d-3233db98adac-kube-api-access-7bwq5\") pod \"migrator-d57d58cc9-2zl8z\" (UID: \"22ed4150-a056-4e6d-be0d-3233db98adac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" Jul 06 14:15:54.814334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.814305 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bwq5\" (UniqueName: \"kubernetes.io/projected/22ed4150-a056-4e6d-be0d-3233db98adac-kube-api-access-7bwq5\") pod \"migrator-d57d58cc9-2zl8z\" (UID: \"22ed4150-a056-4e6d-be0d-3233db98adac\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" Jul 06 14:15:54.949529 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:54.949498 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" Jul 06 14:15:55.397396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:55.397296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:55.397396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:55.397375 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:55.397864 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:55.397402 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:15:55.397864 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:55.397483 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:57.397460713 +0000 UTC m=+73.092300938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : configmap "kube-rbac-proxy" not found Jul 06 14:15:55.397864 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:55.397489 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:15:55.397864 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:55.397540 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:15:57.397525866 +0000 UTC m=+73.092366110 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:15:55.995788 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:55.995760 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sqwhz_135e0dca-d92d-483c-9e81-93ff18d532f4/dns-node-resolver/0.log" Jul 06 14:15:56.653753 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:56.653724 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z"] Jul 06 14:15:56.657015 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:15:56.656984 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ed4150_a056_4e6d_be0d_3233db98adac.slice/crio-27150853412c1fe19169c9df24d7e8ce701e812b0ce0990526e105096d6f6b76 WatchSource:0}: Error finding container 27150853412c1fe19169c9df24d7e8ce701e812b0ce0990526e105096d6f6b76: Status 404 returned error can't find the container with id 27150853412c1fe19169c9df24d7e8ce701e812b0ce0990526e105096d6f6b76 Jul 06 14:15:56.787842 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:56.787774 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sqrlm_4e4b8620-7304-4e70-9f89-434c533ae3f0/node-ca/0.log" Jul 06 14:15:57.111284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.111177 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kx8z5" event={"ID":"aaf11e30-18ff-48e9-9116-bc8b56bdc1c8","Type":"ContainerStarted","Data":"d71cb46836d56957f2b0cfcd2076ba53a30a0d424844b24f09a12fa01801e4b4"} Jul 06 14:15:57.111444 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.111319 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:15:57.112385 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.112359 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" event={"ID":"22ed4150-a056-4e6d-be0d-3233db98adac","Type":"ContainerStarted","Data":"27150853412c1fe19169c9df24d7e8ce701e812b0ce0990526e105096d6f6b76"} Jul 06 14:15:57.113563 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.113540 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mjjwx" event={"ID":"daf0384f-5fef-48ab-a5e9-219455c47099","Type":"ContainerStarted","Data":"35935f7742fdf114f1b57da455d3e61aecdbc9a4dc731ae5166cc0bb5762afc8"} Jul 06 14:15:57.145182 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.145135 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kx8z5" podStartSLOduration=66.740183965 podStartE2EDuration="1m12.145120658s" totalImagesPullingTime="5.404936693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:57.143163663 +0000 UTC m=+72.838003924" watchObservedRunningTime="2026-07-06 14:15:57.145120658 +0000 UTC m=+72.839960904" Jul 06 14:15:57.175021 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.174978 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mjjwx" podStartSLOduration=66.684473226 podStartE2EDuration="1m12.174961917s" totalImagesPullingTime="5.490488691s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:57.174858738 +0000 UTC m=+72.869698986" watchObservedRunningTime="2026-07-06 14:15:57.174961917 +0000 UTC m=+72.869802154" Jul 06 14:15:57.412966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.412883 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:57.412966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.412933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:15:57.413139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:57.413013 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:15:57.413139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:57.413026 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:15:57.413139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:57.413080 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:16:01.413067545 +0000 UTC m=+77.107907771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:15:57.413139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:57.413094 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:16:01.413087229 +0000 UTC m=+77.107927454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : configmap "kube-rbac-proxy" not found Jul 06 14:15:57.585088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:57.585058 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtpmk_685b830e-dc3d-4669-9524-ee547345608a/serve-healthcheck-canary/0.log" Jul 06 14:15:58.148466 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:58.148436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:58.148829 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:58.148518 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:15:58.148876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:58.148835 2581 scope.go:117] "RemoveContainer" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" Jul 06 14:15:58.148986 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:15:58.148969 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-bz6fs_openshift-console-operator(e8f25865-ef65-43b0-849d-bf8786f97e50)\"" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podUID="e8f25865-ef65-43b0-849d-bf8786f97e50" Jul 06 14:15:59.121912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:59.121879 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" event={"ID":"22ed4150-a056-4e6d-be0d-3233db98adac","Type":"ContainerStarted","Data":"cf0bd17efee91e66360acfd7e8b84b30138c13d72a666f53462a7242de066e62"} Jul 06 14:15:59.122080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:59.121931 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" event={"ID":"22ed4150-a056-4e6d-be0d-3233db98adac","Type":"ContainerStarted","Data":"849d514712baacc1341b965237d1de3c9ec30ad2ee27cdb2b91fdcc4a5c7a7ce"} Jul 06 14:15:59.176690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:15:59.176639 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-2zl8z" podStartSLOduration=3.461858717 podStartE2EDuration="5.17662064s" totalImagesPullingTime="1.714761923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:15:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:15:59.166666558 +0000 UTC m=+74.861506802" watchObservedRunningTime="2026-07-06 14:15:59.17662064 +0000 UTC m=+74.871460887" Jul 06 14:16:01.444740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:01.444686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:01.444740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:01.444752 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:01.445225 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:01.444815 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 14:16:01.445225 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:01.444851 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:16:01.445225 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:01.444878 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:16:09.444862862 +0000 UTC m=+85.139703087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : configmap "kube-rbac-proxy" not found Jul 06 14:16:01.445225 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:01.444898 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:16:09.444886118 +0000 UTC m=+85.139726343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:16:09.511554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:09.511408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:09.511554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:09.511473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:09.512139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:09.511611 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 14:16:09.512139 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:09.511687 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls podName:24889622-5265-4405-8061-b2cd283c091f nodeName:}" failed. No retries permitted until 2026-07-06 14:16:25.511664592 +0000 UTC m=+101.206504838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcjlq" (UID: "24889622-5265-4405-8061-b2cd283c091f") : secret "insights-runtime-extractor-tls" not found Jul 06 14:16:09.512139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:09.511914 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/24889622-5265-4405-8061-b2cd283c091f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:12.808942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:12.808908 2581 scope.go:117] "RemoveContainer" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" Jul 06 14:16:13.163559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.163480 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:16:13.163889 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.163874 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/1.log" Jul 06 14:16:13.163951 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.163909 2581 generic.go:356] "Generic (PLEG): container finished" podID="e8f25865-ef65-43b0-849d-bf8786f97e50" containerID="e62f08b45cb6580c8fcdd9a60cad5b44f0b83defc81011b4e8d0d2308d4e45fd" exitCode=255 Jul 06 14:16:13.163997 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.163982 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" event={"ID":"e8f25865-ef65-43b0-849d-bf8786f97e50","Type":"ContainerDied","Data":"e62f08b45cb6580c8fcdd9a60cad5b44f0b83defc81011b4e8d0d2308d4e45fd"} Jul 06 14:16:13.164050 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.164036 2581 scope.go:117] "RemoveContainer" containerID="b18b7f9cfd16176825b7f36e2c42d9de5d24ad647837166f73947a81e53b29ff" Jul 06 14:16:13.164382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:13.164365 2581 scope.go:117] "RemoveContainer" containerID="e62f08b45cb6580c8fcdd9a60cad5b44f0b83defc81011b4e8d0d2308d4e45fd" Jul 06 14:16:13.164554 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:13.164535 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-bz6fs_openshift-console-operator(e8f25865-ef65-43b0-849d-bf8786f97e50)\"" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podUID="e8f25865-ef65-43b0-849d-bf8786f97e50" Jul 06 14:16:14.168568 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:14.168541 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:16:18.148912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:18.148873 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:16:18.149333 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:18.148929 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:16:18.149333 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:18.149237 2581 scope.go:117] "RemoveContainer" containerID="e62f08b45cb6580c8fcdd9a60cad5b44f0b83defc81011b4e8d0d2308d4e45fd" Jul 06 14:16:18.149405 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:18.149378 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-bz6fs_openshift-console-operator(e8f25865-ef65-43b0-849d-bf8786f97e50)\"" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podUID="e8f25865-ef65-43b0-849d-bf8786f97e50" Jul 06 14:16:21.779153 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.779115 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25"] Jul 06 14:16:21.781595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.781579 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.788602 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.788579 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 14:16:21.788732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.788604 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 14:16:21.789644 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.789625 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 14:16:21.789644 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.789643 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 14:16:21.789768 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.789694 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 14:16:21.789768 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.789738 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 14:16:21.789943 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.789931 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 14:16:21.814973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.814944 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25"] Jul 06 14:16:21.899164 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.899348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899225 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.899348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.899452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899349 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.899452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899373 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb22fab0-91f1-49b6-b0e9-bea9926657d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.899452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.899412 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8ptr\" (UniqueName: \"kubernetes.io/projected/fb22fab0-91f1-49b6-b0e9-bea9926657d4-kube-api-access-x8ptr\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:21.999948 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.999911 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000124 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:21.999957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.000123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.000156 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb22fab0-91f1-49b6-b0e9-bea9926657d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.000175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8ptr\" (UniqueName: \"kubernetes.io/projected/fb22fab0-91f1-49b6-b0e9-bea9926657d4-kube-api-access-x8ptr\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000313 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.000206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.000892 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.000839 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb22fab0-91f1-49b6-b0e9-bea9926657d4-additional-service-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.002620 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.002588 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.002936 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.002918 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-ca\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.003128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.003108 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.003232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.003216 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/fb22fab0-91f1-49b6-b0e9-bea9926657d4-hub\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.014336 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.014310 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8ptr\" (UniqueName: \"kubernetes.io/projected/fb22fab0-91f1-49b6-b0e9-bea9926657d4-kube-api-access-x8ptr\") pod \"cluster-proxy-proxy-agent-58cb88cd96-j7n25\" (UID: \"fb22fab0-91f1-49b6-b0e9-bea9926657d4\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.106469 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.106380 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" Jul 06 14:16:22.257740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:22.257714 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25"] Jul 06 14:16:22.260899 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:22.260036 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb22fab0_91f1_49b6_b0e9_bea9926657d4.slice/crio-bacabb5be3dc2aba8b3fdd776fd61881aff97ebcb113262e732b5ac8f0697979 WatchSource:0}: Error finding container bacabb5be3dc2aba8b3fdd776fd61881aff97ebcb113262e732b5ac8f0697979: Status 404 returned error can't find the container with id bacabb5be3dc2aba8b3fdd776fd61881aff97ebcb113262e732b5ac8f0697979 Jul 06 14:16:23.196380 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:23.196339 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" event={"ID":"fb22fab0-91f1-49b6-b0e9-bea9926657d4","Type":"ContainerStarted","Data":"bacabb5be3dc2aba8b3fdd776fd61881aff97ebcb113262e732b5ac8f0697979"} Jul 06 14:16:24.117606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.117566 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:16:24.117804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.117612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:16:24.120066 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.120039 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9769f741-41f9-406f-8361-4164f03fbd8a-metrics-tls\") pod \"dns-default-hz5n9\" (UID: \"9769f741-41f9-406f-8361-4164f03fbd8a\") " pod="openshift-dns/dns-default-hz5n9" Jul 06 14:16:24.120186 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.120168 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"image-registry-74f77d4b8b-sdpbx\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:16:24.142257 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.142228 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-f7sgw\"" Jul 06 14:16:24.150115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.150098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hz5n9" Jul 06 14:16:24.279512 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.279479 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hz5n9"] Jul 06 14:16:24.282273 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:24.282243 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9769f741_41f9_406f_8361_4164f03fbd8a.slice/crio-f31c58755aafb7846b570179d556bb2339b38aa4181f6a24886885d1ea5c6527 WatchSource:0}: Error finding container f31c58755aafb7846b570179d556bb2339b38aa4181f6a24886885d1ea5c6527: Status 404 returned error can't find the container with id f31c58755aafb7846b570179d556bb2339b38aa4181f6a24886885d1ea5c6527 Jul 06 14:16:24.410011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.409935 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gqm6v\"" Jul 06 14:16:24.417872 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.417840 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:16:24.550098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:24.550068 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:16:24.553383 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:24.553358 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc11d03b_01e1_4b5e_bed2_ccce79b70b65.slice/crio-ad08ccbd75462bd7c65247ceb35edc096e87ee0447cb74729756c14f8515c418 WatchSource:0}: Error finding container ad08ccbd75462bd7c65247ceb35edc096e87ee0447cb74729756c14f8515c418: Status 404 returned error can't find the container with id ad08ccbd75462bd7c65247ceb35edc096e87ee0447cb74729756c14f8515c418 Jul 06 14:16:25.203491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.203449 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" event={"ID":"fc11d03b-01e1-4b5e-bed2-ccce79b70b65","Type":"ContainerStarted","Data":"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e"} Jul 06 14:16:25.203670 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.203508 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" event={"ID":"fc11d03b-01e1-4b5e-bed2-ccce79b70b65","Type":"ContainerStarted","Data":"ad08ccbd75462bd7c65247ceb35edc096e87ee0447cb74729756c14f8515c418"} Jul 06 14:16:25.203670 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.203530 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:16:25.204544 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.204522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hz5n9" event={"ID":"9769f741-41f9-406f-8361-4164f03fbd8a","Type":"ContainerStarted","Data":"f31c58755aafb7846b570179d556bb2339b38aa4181f6a24886885d1ea5c6527"} Jul 06 14:16:25.233875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.233824 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" podStartSLOduration=99.233811002 podStartE2EDuration="1m39.233811002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:14:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:25.231839354 +0000 UTC m=+100.926679600" watchObservedRunningTime="2026-07-06 14:16:25.233811002 +0000 UTC m=+100.928651266" Jul 06 14:16:25.530684 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.530600 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:25.533359 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.533329 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/24889622-5265-4405-8061-b2cd283c091f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcjlq\" (UID: \"24889622-5265-4405-8061-b2cd283c091f\") " pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:25.812715 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.812630 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcjlq" Jul 06 14:16:25.958098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:25.957930 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcjlq"] Jul 06 14:16:26.293571 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:26.293538 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24889622_5265_4405_8061_b2cd283c091f.slice/crio-fc7f715bbc3240afeb2272157fa3f9df3d200e02063797f3648b85b0628e1c78 WatchSource:0}: Error finding container fc7f715bbc3240afeb2272157fa3f9df3d200e02063797f3648b85b0628e1c78: Status 404 returned error can't find the container with id fc7f715bbc3240afeb2272157fa3f9df3d200e02063797f3648b85b0628e1c78 Jul 06 14:16:27.221986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:27.221925 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcjlq" event={"ID":"24889622-5265-4405-8061-b2cd283c091f","Type":"ContainerStarted","Data":"fc7f715bbc3240afeb2272157fa3f9df3d200e02063797f3648b85b0628e1c78"} Jul 06 14:16:27.225349 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:27.224735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hz5n9" event={"ID":"9769f741-41f9-406f-8361-4164f03fbd8a","Type":"ContainerStarted","Data":"310085772fa3b61b7d5d7a8118411698faf9b265cec140a796793c7ff9afe90f"} Jul 06 14:16:27.225349 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:27.224784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hz5n9" event={"ID":"9769f741-41f9-406f-8361-4164f03fbd8a","Type":"ContainerStarted","Data":"e79f8f8b8e61a36f81b415af831193b0eccc996153afb4ae15c82f5eb66a4f8f"} Jul 06 14:16:27.225349 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:27.225031 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hz5n9" Jul 06 14:16:27.247560 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:27.247502 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hz5n9" podStartSLOduration=65.191235959 podStartE2EDuration="1m7.247482361s" totalImagesPullingTime="2.056246402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:15:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:27.245868823 +0000 UTC m=+102.940709069" watchObservedRunningTime="2026-07-06 14:16:27.247482361 +0000 UTC m=+102.942322612" Jul 06 14:16:28.119681 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:28.119654 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kx8z5" Jul 06 14:16:31.244010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:31.243968 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcjlq" event={"ID":"24889622-5265-4405-8061-b2cd283c091f","Type":"ContainerStarted","Data":"2d4cd2fb24319d446bca097a5f7198d34b6efe5e434f72d4872c1dedb71c2ff5"} Jul 06 14:16:32.248772 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:32.248735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcjlq" event={"ID":"24889622-5265-4405-8061-b2cd283c091f","Type":"ContainerStarted","Data":"8f183a88cb5235ab3bb82a3b6c7df874cb39b308a117e77c3a9df904bb9c2a56"} Jul 06 14:16:33.351165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.351127 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tqlhq"] Jul 06 14:16:33.353851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.353828 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.358849 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.358819 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 14:16:33.358981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.358876 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 14:16:33.359137 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.359118 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 14:16:33.359281 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.359220 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 14:16:33.360335 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.360313 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hhpsl\"" Jul 06 14:16:33.360469 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.360393 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 14:16:33.362913 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.362895 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 14:16:33.496170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496131 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496177 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496223 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-accelerators-collector-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-wtmp\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496291 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zb6f\" (UniqueName: \"kubernetes.io/projected/73a7b71e-6e3f-4861-b509-da2c6bfc2448-kube-api-access-2zb6f\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496320 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-metrics-client-ca\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496344 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-sys\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496607 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-textfile\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.496607 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.496512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-root\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597694 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-textfile\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597714 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-root\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-accelerators-collector-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.597872 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-root\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598142 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:33.598023 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 14:16:33.598142 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598069 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-textfile\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598142 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598072 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-wtmp\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598662 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:16:33.598527 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls podName:73a7b71e-6e3f-4861-b509-da2c6bfc2448 nodeName:}" failed. No retries permitted until 2026-07-06 14:16:34.098081146 +0000 UTC m=+109.792921391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls") pod "node-exporter-tqlhq" (UID: "73a7b71e-6e3f-4861-b509-da2c6bfc2448") : secret "node-exporter-tls" not found Jul 06 14:16:33.598662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598551 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-wtmp\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zb6f\" (UniqueName: \"kubernetes.io/projected/73a7b71e-6e3f-4861-b509-da2c6bfc2448-kube-api-access-2zb6f\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598697 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-metrics-client-ca\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598730 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-sys\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.598884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598816 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73a7b71e-6e3f-4861-b509-da2c6bfc2448-sys\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.601000 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.598903 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-accelerators-collector-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.601000 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.599587 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73a7b71e-6e3f-4861-b509-da2c6bfc2448-metrics-client-ca\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.603072 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.602736 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.621505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.621478 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zb6f\" (UniqueName: \"kubernetes.io/projected/73a7b71e-6e3f-4861-b509-da2c6bfc2448-kube-api-access-2zb6f\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:33.808715 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:33.808688 2581 scope.go:117] "RemoveContainer" containerID="e62f08b45cb6580c8fcdd9a60cad5b44f0b83defc81011b4e8d0d2308d4e45fd" Jul 06 14:16:34.104452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:34.104357 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:34.107211 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:34.107163 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73a7b71e-6e3f-4861-b509-da2c6bfc2448-node-exporter-tls\") pod \"node-exporter-tqlhq\" (UID: \"73a7b71e-6e3f-4861-b509-da2c6bfc2448\") " pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:34.264728 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:34.264693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tqlhq" Jul 06 14:16:34.415698 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:34.415670 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a7b71e_6e3f_4861_b509_da2c6bfc2448.slice/crio-891944679cc3968426383c8fcbf2e3d30f040986bd3a7d3217d3a89f48b4038f WatchSource:0}: Error finding container 891944679cc3968426383c8fcbf2e3d30f040986bd3a7d3217d3a89f48b4038f: Status 404 returned error can't find the container with id 891944679cc3968426383c8fcbf2e3d30f040986bd3a7d3217d3a89f48b4038f Jul 06 14:16:35.260248 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.260177 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tqlhq" event={"ID":"73a7b71e-6e3f-4861-b509-da2c6bfc2448","Type":"ContainerStarted","Data":"891944679cc3968426383c8fcbf2e3d30f040986bd3a7d3217d3a89f48b4038f"} Jul 06 14:16:35.261896 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.261873 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:16:35.262009 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.261986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" event={"ID":"e8f25865-ef65-43b0-849d-bf8786f97e50","Type":"ContainerStarted","Data":"23f67c6336f36db9c994d350bbbea706c91213bc1c4dd7545ef045c48c5e00f3"} Jul 06 14:16:35.262353 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.262302 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:16:35.264657 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.264628 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcjlq" event={"ID":"24889622-5265-4405-8061-b2cd283c091f","Type":"ContainerStarted","Data":"fe77d2f30c1e0905383f8687aa59add77212407245ebe2996187c8e05bf5fe7f"} Jul 06 14:16:35.267606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.267584 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" Jul 06 14:16:35.297713 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.297664 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-bz6fs" podStartSLOduration=45.757385519 podStartE2EDuration="48.297648744s" totalImagesPullingTime="2.540263225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:35.295776016 +0000 UTC m=+110.990616266" watchObservedRunningTime="2026-07-06 14:16:35.297648744 +0000 UTC m=+110.992488990" Jul 06 14:16:35.330112 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:35.330052 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hcjlq" podStartSLOduration=34.327424881 podStartE2EDuration="42.330029187s" totalImagesPullingTime="8.002604306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:15:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:35.329187921 +0000 UTC m=+111.024028168" watchObservedRunningTime="2026-07-06 14:16:35.330029187 +0000 UTC m=+111.024869437" Jul 06 14:16:36.269270 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:36.269228 2581 generic.go:356] "Generic (PLEG): container finished" podID="73a7b71e-6e3f-4861-b509-da2c6bfc2448" containerID="c72f790db479202c8a3d75aadb3dcb8e2f3fec5459c99f5d58a80821918fe3a5" exitCode=0 Jul 06 14:16:36.269732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:36.269328 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tqlhq" event={"ID":"73a7b71e-6e3f-4861-b509-da2c6bfc2448","Type":"ContainerDied","Data":"c72f790db479202c8a3d75aadb3dcb8e2f3fec5459c99f5d58a80821918fe3a5"} Jul 06 14:16:37.234513 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:37.234474 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hz5n9" Jul 06 14:16:37.277211 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:37.277150 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tqlhq" event={"ID":"73a7b71e-6e3f-4861-b509-da2c6bfc2448","Type":"ContainerStarted","Data":"c79d9d348dc93e452f235946ab4293afbeb6471faf13523a8b50c30884628d64"} Jul 06 14:16:37.277672 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:37.277223 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tqlhq" event={"ID":"73a7b71e-6e3f-4861-b509-da2c6bfc2448","Type":"ContainerStarted","Data":"66a9da542cd81b86acbed7b2e370dac6b22a7a5b1627f7136e5bf54b88665266"} Jul 06 14:16:37.327973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:37.327922 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tqlhq" podStartSLOduration=3.186621794 podStartE2EDuration="4.327905292s" totalImagesPullingTime="1.069384398s" totalInitContainerRuntime="71.8991ms" isStatefulPod=true podCreationTimestamp="2026-07-06 14:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:37.323003447 +0000 UTC m=+113.017843694" watchObservedRunningTime="2026-07-06 14:16:37.327905292 +0000 UTC m=+113.022745594" Jul 06 14:16:38.492723 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.492690 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-59dbdc5959-b72qt"] Jul 06 14:16:38.494918 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.494903 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.502438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.502411 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 14:16:38.502559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.502501 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 14:16:38.502559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.502513 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 14:16:38.502654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.502591 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-82wtb\"" Jul 06 14:16:38.502831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.502818 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 14:16:38.503926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.503909 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 14:16:38.509377 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.509358 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 14:16:38.523978 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.523941 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59dbdc5959-b72qt"] Jul 06 14:16:38.642541 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642541 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-serving-certs-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642740 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642720 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s96cg\" (UniqueName: \"kubernetes.io/projected/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-kube-api-access-s96cg\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642833 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-federate-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642833 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642780 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.642833 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.642807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-metrics-client-ca\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.743913 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.743832 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.743913 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.743889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-metrics-client-ca\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.743918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.743940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-serving-certs-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.743955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.744000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.744028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s96cg\" (UniqueName: \"kubernetes.io/projected/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-kube-api-access-s96cg\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.744110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.744053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-federate-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.745073 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.745042 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-serving-certs-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.745213 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.745138 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-metrics-client-ca\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.745323 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.745303 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.746721 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.746699 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.746819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.746721 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.746970 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.746951 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-federate-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.747080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.747061 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-telemeter-client-tls\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.756578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.756559 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s96cg\" (UniqueName: \"kubernetes.io/projected/6ee90ea7-9e8b-405e-a4bd-3759573e62eb-kube-api-access-s96cg\") pod \"telemeter-client-59dbdc5959-b72qt\" (UID: \"6ee90ea7-9e8b-405e-a4bd-3759573e62eb\") " pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.803494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.803462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" Jul 06 14:16:38.986450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:38.986427 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-59dbdc5959-b72qt"] Jul 06 14:16:38.988256 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:16:38.988233 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee90ea7_9e8b_405e_a4bd_3759573e62eb.slice/crio-0acbbc97ea39aa1ef90040527a41b556b5be4731c2803b48ce32be26fd8d4d5c WatchSource:0}: Error finding container 0acbbc97ea39aa1ef90040527a41b556b5be4731c2803b48ce32be26fd8d4d5c: Status 404 returned error can't find the container with id 0acbbc97ea39aa1ef90040527a41b556b5be4731c2803b48ce32be26fd8d4d5c Jul 06 14:16:39.284942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:39.284910 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" event={"ID":"6ee90ea7-9e8b-405e-a4bd-3759573e62eb","Type":"ContainerStarted","Data":"0acbbc97ea39aa1ef90040527a41b556b5be4731c2803b48ce32be26fd8d4d5c"} Jul 06 14:16:42.297369 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:42.297327 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" event={"ID":"6ee90ea7-9e8b-405e-a4bd-3759573e62eb","Type":"ContainerStarted","Data":"a3723816e534268790aa69ea5bdd1a4e2227b1bb40dbb1f65b305b0050b1434b"} Jul 06 14:16:43.302845 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:43.302811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" event={"ID":"6ee90ea7-9e8b-405e-a4bd-3759573e62eb","Type":"ContainerStarted","Data":"d7acd1d5062247e577a6fb6719e31a5d037b262601194aa7ab49b4e5d9cdbd51"} Jul 06 14:16:43.303365 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:43.302860 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" event={"ID":"6ee90ea7-9e8b-405e-a4bd-3759573e62eb","Type":"ContainerStarted","Data":"9e926b1887ccbdc7de252725e74a4d381781577e764daeba4e82776d2e42a005"} Jul 06 14:16:43.356405 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:43.356347 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-59dbdc5959-b72qt" podStartSLOduration=1.9606004289999999 podStartE2EDuration="5.35633283s" totalImagesPullingTime="3.395732401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:16:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:43.355803293 +0000 UTC m=+119.050643539" watchObservedRunningTime="2026-07-06 14:16:43.35633283 +0000 UTC m=+119.051173076" Jul 06 14:16:44.132769 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:44.129873 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:16:44.137008 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:44.136982 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:16:49.325500 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:49.325464 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" event={"ID":"fb22fab0-91f1-49b6-b0e9-bea9926657d4","Type":"ContainerStarted","Data":"c6c0615b9daeababdbecb8e71bfc0f277de2804c4b8ac43aecbbff7aaf0d1296"} Jul 06 14:16:49.325886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:49.325510 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" event={"ID":"fb22fab0-91f1-49b6-b0e9-bea9926657d4","Type":"ContainerStarted","Data":"dcec03468d2c39a1ae379e62b4101fe9970bd699d87c04ad096eace9821d94f6"} Jul 06 14:16:49.325886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:49.325534 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" event={"ID":"fb22fab0-91f1-49b6-b0e9-bea9926657d4","Type":"ContainerStarted","Data":"2b76c1a9a767a1392e6745193f587a13982e62e1f7e159918026f72abec79e52"} Jul 06 14:16:49.351558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:49.351507 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-58cb88cd96-j7n25" podStartSLOduration=2.279743394 podStartE2EDuration="28.351493015s" totalImagesPullingTime="26.071749621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:49.350017572 +0000 UTC m=+125.044857843" watchObservedRunningTime="2026-07-06 14:16:49.351493015 +0000 UTC m=+125.046333262" Jul 06 14:16:54.471945 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:54.471896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:16:54.474448 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:54.474424 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d4e36c94-f9aa-45b2-b1f6-90974c8dca06-metrics-certs\") pod \"network-metrics-daemon-rkfvm\" (UID: \"d4e36c94-f9aa-45b2-b1f6-90974c8dca06\") " pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:16:54.527775 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:54.527746 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mlf5t\"" Jul 06 14:16:54.534596 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:54.534575 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkfvm" Jul 06 14:16:54.661652 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:54.661619 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkfvm"] Jul 06 14:16:55.344540 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:55.344506 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkfvm" event={"ID":"d4e36c94-f9aa-45b2-b1f6-90974c8dca06","Type":"ContainerStarted","Data":"aa52d63e8096fbbd262c78a7ddfeb19993bf92e54cbd1db04dc178712ff1d6d4"} Jul 06 14:16:56.349796 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:56.349698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkfvm" event={"ID":"d4e36c94-f9aa-45b2-b1f6-90974c8dca06","Type":"ContainerStarted","Data":"1b8d62027883f4f106bd05f3d99ea91d5b5ce96589874a8ad598ff13ce996272"} Jul 06 14:16:56.349796 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:56.349753 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkfvm" event={"ID":"d4e36c94-f9aa-45b2-b1f6-90974c8dca06","Type":"ContainerStarted","Data":"6df98b228f954b472d0bff6409ea1a1330bc34ce3a39d920768869b6ed881ac8"} Jul 06 14:16:56.373297 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:16:56.373244 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rkfvm" podStartSLOduration=130.064109123 podStartE2EDuration="2m11.373229556s" totalImagesPullingTime="1.309120433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:16:56.37201219 +0000 UTC m=+132.066852437" watchObservedRunningTime="2026-07-06 14:16:56.373229556 +0000 UTC m=+132.068069803" Jul 06 14:17:09.152638 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.152593 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" podUID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" containerName="registry" containerID="cri-o://3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e" gracePeriod=30 Jul 06 14:17:09.388025 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.387998 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:17:09.393254 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.393173 2581 generic.go:356] "Generic (PLEG): container finished" podID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" containerID="3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e" exitCode=0 Jul 06 14:17:09.393352 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.393222 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" event={"ID":"fc11d03b-01e1-4b5e-bed2-ccce79b70b65","Type":"ContainerDied","Data":"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e"} Jul 06 14:17:09.393390 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.393348 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" event={"ID":"fc11d03b-01e1-4b5e-bed2-ccce79b70b65","Type":"ContainerDied","Data":"ad08ccbd75462bd7c65247ceb35edc096e87ee0447cb74729756c14f8515c418"} Jul 06 14:17:09.393390 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.393370 2581 scope.go:117] "RemoveContainer" containerID="3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e" Jul 06 14:17:09.393390 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.393233 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f77d4b8b-sdpbx" Jul 06 14:17:09.400964 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.400942 2581 scope.go:117] "RemoveContainer" containerID="3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e" Jul 06 14:17:09.401250 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:17:09.401231 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e\": container with ID starting with 3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e not found: ID does not exist" containerID="3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e" Jul 06 14:17:09.401328 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.401262 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e"} err="failed to get container status \"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e\": rpc error: code = NotFound desc = could not find container \"3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e\": container with ID starting with 3e104d84625a38455cc2747a9a8a60d7d80c0ecb5c26f9cc2d77711a15fab98e not found: ID does not exist" Jul 06 14:17:09.490692 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490660 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.490868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490710 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.490868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490742 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.490868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490776 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.490868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490810 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.490868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5c2p\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.491109 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490895 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.491109 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.490946 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca\") pod \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\" (UID: \"fc11d03b-01e1-4b5e-bed2-ccce79b70b65\") " Jul 06 14:17:09.491827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.491786 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:17:09.491827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.491797 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:17:09.493456 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.493417 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:17:09.493583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.493546 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p" (OuterVolumeSpecName: "kube-api-access-l5c2p") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "kube-api-access-l5c2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:17:09.493690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.493671 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:17:09.493779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.493762 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:17:09.493874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.493851 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:17:09.499662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.499637 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fc11d03b-01e1-4b5e-bed2-ccce79b70b65" (UID: "fc11d03b-01e1-4b5e-bed2-ccce79b70b65"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:17:09.591630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591596 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-ca-trust-extracted\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591623 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5c2p\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-kube-api-access-l5c2p\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591632 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-certificates\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591642 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-trusted-ca\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591650 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-installation-pull-secrets\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591658 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-bound-sa-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591668 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-image-registry-private-configuration\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.591893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.591677 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fc11d03b-01e1-4b5e-bed2-ccce79b70b65-registry-tls\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:17:09.729351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.729322 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:17:09.734083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:09.734055 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74f77d4b8b-sdpbx"] Jul 06 14:17:10.812873 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:17:10.812836 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" path="/var/lib/kubelet/pods/fc11d03b-01e1-4b5e-bed2-ccce79b70b65/volumes" Jul 06 14:18:35.073443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.073407 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2"] Jul 06 14:18:35.078264 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.078228 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" containerName="registry" Jul 06 14:18:35.078428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.078305 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" containerName="registry" Jul 06 14:18:35.078470 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.078426 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc11d03b-01e1-4b5e-bed2-ccce79b70b65" containerName="registry" Jul 06 14:18:35.081521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.081506 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.088945 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.088922 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:18:35.089053 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.088976 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:18:35.090461 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.090445 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5g2bj\"" Jul 06 14:18:35.099742 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.099722 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2"] Jul 06 14:18:35.178900 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.178860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.179063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.178918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.179063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.178972 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs2wt\" (UniqueName: \"kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.279575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.279539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs2wt\" (UniqueName: \"kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.279748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.279592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.279748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.279615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.279947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.279930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.280001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.279987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.295454 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.295425 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs2wt\" (UniqueName: \"kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.391080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.391001 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:35.536981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.536953 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2"] Jul 06 14:18:35.540495 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:18:35.540466 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9d7835_74bc_4577_bd6b_785c7f61fc65.slice/crio-dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf WatchSource:0}: Error finding container dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf: Status 404 returned error can't find the container with id dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf Jul 06 14:18:35.677307 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:35.677221 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" event={"ID":"cd9d7835-74bc-4577-bd6b-785c7f61fc65","Type":"ContainerStarted","Data":"dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf"} Jul 06 14:18:44.708136 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:44.708102 2581 generic.go:356] "Generic (PLEG): container finished" podID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerID="e970ce6b04165e64c92e314f9327394cffdcd6789cc4891bed89d33e21d59efd" exitCode=0 Jul 06 14:18:44.708508 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:44.708173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" event={"ID":"cd9d7835-74bc-4577-bd6b-785c7f61fc65","Type":"ContainerDied","Data":"e970ce6b04165e64c92e314f9327394cffdcd6789cc4891bed89d33e21d59efd"} Jul 06 14:18:46.716789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:46.716754 2581 generic.go:356] "Generic (PLEG): container finished" podID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerID="33b36a5791c3329d15ac23eddd1c39fc942a189f9c5316432f39f4c1a1c59547" exitCode=0 Jul 06 14:18:46.716789 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:46.716792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" event={"ID":"cd9d7835-74bc-4577-bd6b-785c7f61fc65","Type":"ContainerDied","Data":"33b36a5791c3329d15ac23eddd1c39fc942a189f9c5316432f39f4c1a1c59547"} Jul 06 14:18:55.752715 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:55.752677 2581 generic.go:356] "Generic (PLEG): container finished" podID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerID="58a721438346040253b4b5da29a2d3e87edc20bfd32bfb684e55bc41b082db66" exitCode=0 Jul 06 14:18:55.753102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:55.752722 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" event={"ID":"cd9d7835-74bc-4577-bd6b-785c7f61fc65","Type":"ContainerDied","Data":"58a721438346040253b4b5da29a2d3e87edc20bfd32bfb684e55bc41b082db66"} Jul 06 14:18:56.884295 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.884270 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:18:56.938123 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.938094 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rs2wt\" (UniqueName: \"kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt\") pod \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " Jul 06 14:18:56.938327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.938148 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle\") pod \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " Jul 06 14:18:56.938379 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.938327 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util\") pod \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\" (UID: \"cd9d7835-74bc-4577-bd6b-785c7f61fc65\") " Jul 06 14:18:56.938883 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.938824 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle" (OuterVolumeSpecName: "bundle") pod "cd9d7835-74bc-4577-bd6b-785c7f61fc65" (UID: "cd9d7835-74bc-4577-bd6b-785c7f61fc65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:18:56.940513 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.940479 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt" (OuterVolumeSpecName: "kube-api-access-rs2wt") pod "cd9d7835-74bc-4577-bd6b-785c7f61fc65" (UID: "cd9d7835-74bc-4577-bd6b-785c7f61fc65"). InnerVolumeSpecName "kube-api-access-rs2wt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:18:56.942372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:56.942349 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util" (OuterVolumeSpecName: "util") pod "cd9d7835-74bc-4577-bd6b-785c7f61fc65" (UID: "cd9d7835-74bc-4577-bd6b-785c7f61fc65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:18:57.039168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.039087 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:18:57.039168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.039117 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs2wt\" (UniqueName: \"kubernetes.io/projected/cd9d7835-74bc-4577-bd6b-785c7f61fc65-kube-api-access-rs2wt\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:18:57.039168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.039128 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd9d7835-74bc-4577-bd6b-785c7f61fc65-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:18:57.762049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.762017 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" event={"ID":"cd9d7835-74bc-4577-bd6b-785c7f61fc65","Type":"ContainerDied","Data":"dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf"} Jul 06 14:18:57.762233 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.762063 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd2fb13e9014a14d582eb11724041a597fe38f5e3ea0d2023368135c718c7daf" Jul 06 14:18:57.762233 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:18:57.762032 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540f2df2" Jul 06 14:19:03.575911 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.575874 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg"] Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576101 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="pull" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576112 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="pull" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576123 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="extract" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576128 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="extract" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576136 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="util" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576143 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="util" Jul 06 14:19:03.576348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.576187 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd9d7835-74bc-4577-bd6b-785c7f61fc65" containerName="extract" Jul 06 14:19:03.579855 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.579837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.582930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.582907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tqf6r\"" Jul 06 14:19:03.583389 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.583370 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 14:19:03.583625 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.583612 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:19:03.597637 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.597613 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg"] Jul 06 14:19:03.682710 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.682674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6j6\" (UniqueName: \"kubernetes.io/projected/da01bae8-3d2d-4734-8d9b-816aa417ce95-kube-api-access-tt6j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.682874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.682727 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da01bae8-3d2d-4734-8d9b-816aa417ce95-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.783090 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.783059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6j6\" (UniqueName: \"kubernetes.io/projected/da01bae8-3d2d-4734-8d9b-816aa417ce95-kube-api-access-tt6j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.783263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.783103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da01bae8-3d2d-4734-8d9b-816aa417ce95-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.783458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.783443 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/da01bae8-3d2d-4734-8d9b-816aa417ce95-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.796470 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.796447 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6j6\" (UniqueName: \"kubernetes.io/projected/da01bae8-3d2d-4734-8d9b-816aa417ce95-kube-api-access-tt6j6\") pod \"cert-manager-operator-controller-manager-55c444bf86-88nkg\" (UID: \"da01bae8-3d2d-4734-8d9b-816aa417ce95\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:03.888798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:03.888726 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" Jul 06 14:19:04.039975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:04.039952 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg"] Jul 06 14:19:04.042831 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:04.042792 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda01bae8_3d2d_4734_8d9b_816aa417ce95.slice/crio-73ca8e6de72f984564088656c00e3c766073db9d7540badd55ad22ef255e35b1 WatchSource:0}: Error finding container 73ca8e6de72f984564088656c00e3c766073db9d7540badd55ad22ef255e35b1: Status 404 returned error can't find the container with id 73ca8e6de72f984564088656c00e3c766073db9d7540badd55ad22ef255e35b1 Jul 06 14:19:04.788351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:04.788310 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" event={"ID":"da01bae8-3d2d-4734-8d9b-816aa417ce95","Type":"ContainerStarted","Data":"73ca8e6de72f984564088656c00e3c766073db9d7540badd55ad22ef255e35b1"} Jul 06 14:19:05.798997 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:05.798963 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" event={"ID":"da01bae8-3d2d-4734-8d9b-816aa417ce95","Type":"ContainerStarted","Data":"aa36b90e1223be105fe2d48a04f94ecd83068ad94393c6c75992f4ac4de054fe"} Jul 06 14:19:05.840303 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:05.840239 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-88nkg" podStartSLOduration=1.163843272 podStartE2EDuration="2.840217994s" totalImagesPullingTime="1.676374722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:05.839652063 +0000 UTC m=+261.534492310" watchObservedRunningTime="2026-07-06 14:19:05.840217994 +0000 UTC m=+261.535058237" Jul 06 14:19:07.380978 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.380944 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx"] Jul 06 14:19:07.383303 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.383282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.387126 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.387106 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:19:07.387464 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.387446 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5g2bj\"" Jul 06 14:19:07.388885 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.388866 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:19:07.398753 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.398728 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx"] Jul 06 14:19:07.510873 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.510838 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbsqb\" (UniqueName: \"kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.511056 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.510894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.511056 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.510915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.611492 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.611458 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbsqb\" (UniqueName: \"kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.611651 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.611519 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.611651 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.611535 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.611886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.611870 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.611946 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.611929 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.633178 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.633099 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbsqb\" (UniqueName: \"kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.692827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.692779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:07.834126 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:07.834091 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx"] Jul 06 14:19:07.838053 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:07.838024 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa0bd74f_686a_4793_9de7_ca49256e3b38.slice/crio-12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6 WatchSource:0}: Error finding container 12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6: Status 404 returned error can't find the container with id 12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6 Jul 06 14:19:08.812426 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:08.812394 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerID="4ec763a4d8ee5c44c17825855e60e338c04cbd2848589486b0999e6bb0119353" exitCode=0 Jul 06 14:19:08.812782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:08.812637 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" event={"ID":"aa0bd74f-686a-4793-9de7-ca49256e3b38","Type":"ContainerDied","Data":"4ec763a4d8ee5c44c17825855e60e338c04cbd2848589486b0999e6bb0119353"} Jul 06 14:19:08.812782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:08.812673 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" event={"ID":"aa0bd74f-686a-4793-9de7-ca49256e3b38","Type":"ContainerStarted","Data":"12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6"} Jul 06 14:19:11.827114 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:11.827083 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerID="491412bfdceaeef4fd0f1cfa1f62d97daaf3968d886c3b6a0d03ba492263f1c0" exitCode=0 Jul 06 14:19:11.827526 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:11.827144 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" event={"ID":"aa0bd74f-686a-4793-9de7-ca49256e3b38","Type":"ContainerDied","Data":"491412bfdceaeef4fd0f1cfa1f62d97daaf3968d886c3b6a0d03ba492263f1c0"} Jul 06 14:19:12.833455 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:12.833422 2581 generic.go:356] "Generic (PLEG): container finished" podID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerID="6702c9c6890c5df6f5e50f7476834c7f2d092267111bf461780a496a46bcc983" exitCode=0 Jul 06 14:19:12.833831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:12.833493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" event={"ID":"aa0bd74f-686a-4793-9de7-ca49256e3b38","Type":"ContainerDied","Data":"6702c9c6890c5df6f5e50f7476834c7f2d092267111bf461780a496a46bcc983"} Jul 06 14:19:13.958816 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:13.958792 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:14.062270 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.062229 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle\") pod \"aa0bd74f-686a-4793-9de7-ca49256e3b38\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " Jul 06 14:19:14.062486 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.062294 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbsqb\" (UniqueName: \"kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb\") pod \"aa0bd74f-686a-4793-9de7-ca49256e3b38\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " Jul 06 14:19:14.062486 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.062327 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util\") pod \"aa0bd74f-686a-4793-9de7-ca49256e3b38\" (UID: \"aa0bd74f-686a-4793-9de7-ca49256e3b38\") " Jul 06 14:19:14.062749 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.062724 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle" (OuterVolumeSpecName: "bundle") pod "aa0bd74f-686a-4793-9de7-ca49256e3b38" (UID: "aa0bd74f-686a-4793-9de7-ca49256e3b38"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:19:14.064590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.064562 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb" (OuterVolumeSpecName: "kube-api-access-fbsqb") pod "aa0bd74f-686a-4793-9de7-ca49256e3b38" (UID: "aa0bd74f-686a-4793-9de7-ca49256e3b38"). InnerVolumeSpecName "kube-api-access-fbsqb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:19:14.067109 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.067087 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util" (OuterVolumeSpecName: "util") pod "aa0bd74f-686a-4793-9de7-ca49256e3b38" (UID: "aa0bd74f-686a-4793-9de7-ca49256e3b38"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:19:14.163577 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.163498 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:14.163577 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.163524 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aa0bd74f-686a-4793-9de7-ca49256e3b38-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:14.163577 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.163533 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbsqb\" (UniqueName: \"kubernetes.io/projected/aa0bd74f-686a-4793-9de7-ca49256e3b38-kube-api-access-fbsqb\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:14.841626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.841603 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" Jul 06 14:19:14.841772 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.841602 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f7l6vx" event={"ID":"aa0bd74f-686a-4793-9de7-ca49256e3b38","Type":"ContainerDied","Data":"12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6"} Jul 06 14:19:14.841772 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:14.841732 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12790bd6ef00c72c8382b6955eca60a79823285cb851ff0e3159ad9f0536f2b6" Jul 06 14:19:19.795094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795059 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-68xtl"] Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795354 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="util" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795366 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="util" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795381 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="pull" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795386 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="pull" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795395 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="extract" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795400 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="extract" Jul 06 14:19:19.795522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.795446 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa0bd74f-686a-4793-9de7-ca49256e3b38" containerName="extract" Jul 06 14:19:19.798539 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.798524 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:19.804777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.804758 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-kc8fb\"" Jul 06 14:19:19.804777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.804769 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 14:19:19.805878 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.805865 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 14:19:19.828736 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.828701 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-68xtl"] Jul 06 14:19:19.902983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.902937 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwmv\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-kube-api-access-8mwmv\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:19.903157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:19.903051 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.004152 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.004116 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.004354 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.004185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwmv\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-kube-api-access-8mwmv\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.022398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.022370 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-bound-sa-token\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.022547 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.022449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwmv\" (UniqueName: \"kubernetes.io/projected/5944862b-5c78-406d-bbcb-679223f240d2-kube-api-access-8mwmv\") pod \"cert-manager-55b4d954d5-68xtl\" (UID: \"5944862b-5c78-406d-bbcb-679223f240d2\") " pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.106558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.106469 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-68xtl" Jul 06 14:19:20.244765 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.244742 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-68xtl"] Jul 06 14:19:20.246766 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:20.246729 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5944862b_5c78_406d_bbcb_679223f240d2.slice/crio-3c353e0feeb1fe2367d7a26b656e66c20077159f81c589d597d83a31b4d7bc1b WatchSource:0}: Error finding container 3c353e0feeb1fe2367d7a26b656e66c20077159f81c589d597d83a31b4d7bc1b: Status 404 returned error can't find the container with id 3c353e0feeb1fe2367d7a26b656e66c20077159f81c589d597d83a31b4d7bc1b Jul 06 14:19:20.863383 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:20.863343 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-68xtl" event={"ID":"5944862b-5c78-406d-bbcb-679223f240d2","Type":"ContainerStarted","Data":"3c353e0feeb1fe2367d7a26b656e66c20077159f81c589d597d83a31b4d7bc1b"} Jul 06 14:19:21.087127 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.087093 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7"] Jul 06 14:19:21.090799 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.090757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.097186 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.097158 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 14:19:21.098835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.098496 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 14:19:21.098835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.098541 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bx7nh\"" Jul 06 14:19:21.109806 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.109782 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7"] Jul 06 14:19:21.213509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.213468 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9wdp\" (UniqueName: \"kubernetes.io/projected/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-kube-api-access-s9wdp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.213689 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.213530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.314634 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.314593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9wdp\" (UniqueName: \"kubernetes.io/projected/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-kube-api-access-s9wdp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.314833 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.314658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.315129 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.315103 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.329188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.329156 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9wdp\" (UniqueName: \"kubernetes.io/projected/fed7ccb7-c0c9-4e77-8a91-125e3ad9aead-kube-api-access-s9wdp\") pod \"openshift-lws-operator-fd8ccff4c-s6rm7\" (UID: \"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.402016 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.401977 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" Jul 06 14:19:21.567096 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.567057 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7"] Jul 06 14:19:21.572738 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:21.572690 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfed7ccb7_c0c9_4e77_8a91_125e3ad9aead.slice/crio-09a395987a0ae07eefdb53fe348c8cf5a15f6139fe224ecd9188d1edf890de26 WatchSource:0}: Error finding container 09a395987a0ae07eefdb53fe348c8cf5a15f6139fe224ecd9188d1edf890de26: Status 404 returned error can't find the container with id 09a395987a0ae07eefdb53fe348c8cf5a15f6139fe224ecd9188d1edf890de26 Jul 06 14:19:21.869832 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:21.869740 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" event={"ID":"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead","Type":"ContainerStarted","Data":"09a395987a0ae07eefdb53fe348c8cf5a15f6139fe224ecd9188d1edf890de26"} Jul 06 14:19:23.879841 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:23.879805 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-68xtl" event={"ID":"5944862b-5c78-406d-bbcb-679223f240d2","Type":"ContainerStarted","Data":"7076fd32a8b34227bc14cbe3916ca3f5208f8fe365106cb44b767fc3f4669b52"} Jul 06 14:19:23.912718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:23.912660 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-68xtl" podStartSLOduration=2.25863139 podStartE2EDuration="4.912639697s" totalImagesPullingTime="2.654008307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:23.912269555 +0000 UTC m=+279.607109803" watchObservedRunningTime="2026-07-06 14:19:23.912639697 +0000 UTC m=+279.607479944" Jul 06 14:19:24.885271 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:24.885232 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" event={"ID":"fed7ccb7-c0c9-4e77-8a91-125e3ad9aead","Type":"ContainerStarted","Data":"b5593c0c92c75ba2c1e9e99b10dc1f2cb1c446e9bd586485c56df60390d22bd4"} Jul 06 14:19:24.918416 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:24.918368 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s6rm7" podStartSLOduration=1.394333953 podStartE2EDuration="3.918351449s" totalImagesPullingTime="2.524017496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:24.918180361 +0000 UTC m=+280.613020611" watchObservedRunningTime="2026-07-06 14:19:24.918351449 +0000 UTC m=+280.613191695" Jul 06 14:19:27.358154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.358119 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8"] Jul 06 14:19:27.376173 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.376139 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8"] Jul 06 14:19:27.376354 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.376308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.379799 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.379777 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5g2bj\"" Jul 06 14:19:27.379979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.379952 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:19:27.381331 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.381312 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:19:27.466462 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.466422 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.466630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.466464 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgnhq\" (UniqueName: \"kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.466630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.466534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.567523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.567486 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.567690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.567527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgnhq\" (UniqueName: \"kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.567690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.567569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.567893 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.567876 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.567933 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.567909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.580827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.580790 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgnhq\" (UniqueName: \"kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.688237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.688107 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:27.849770 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.849746 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8"] Jul 06 14:19:27.851876 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:27.851851 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca87035_0e03_4a45_ab00_bf9cf0006eaf.slice/crio-16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c WatchSource:0}: Error finding container 16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c: Status 404 returned error can't find the container with id 16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c Jul 06 14:19:27.898548 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:27.898512 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" event={"ID":"6ca87035-0e03-4a45-ab00-bf9cf0006eaf","Type":"ContainerStarted","Data":"16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c"} Jul 06 14:19:28.903858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:28.903770 2581 generic.go:356] "Generic (PLEG): container finished" podID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerID="b028f129d3606742ac596145fdd885a17d7bf2b1b42d764b4a164e888b4d2a38" exitCode=0 Jul 06 14:19:28.903858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:28.903814 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" event={"ID":"6ca87035-0e03-4a45-ab00-bf9cf0006eaf","Type":"ContainerDied","Data":"b028f129d3606742ac596145fdd885a17d7bf2b1b42d764b4a164e888b4d2a38"} Jul 06 14:19:29.910130 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:29.910096 2581 generic.go:356] "Generic (PLEG): container finished" podID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerID="9010f5fc06d4594200e97cda533e5307183e2a53113994ac87108b26c525baf4" exitCode=0 Jul 06 14:19:29.910580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:29.910204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" event={"ID":"6ca87035-0e03-4a45-ab00-bf9cf0006eaf","Type":"ContainerDied","Data":"9010f5fc06d4594200e97cda533e5307183e2a53113994ac87108b26c525baf4"} Jul 06 14:19:30.915978 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:30.915937 2581 generic.go:356] "Generic (PLEG): container finished" podID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerID="f4fe01d85825925f3ca4afd0775da3b7ab4b765bedaeffd89dd7ac70db458511" exitCode=0 Jul 06 14:19:30.916388 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:30.916023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" event={"ID":"6ca87035-0e03-4a45-ab00-bf9cf0006eaf","Type":"ContainerDied","Data":"f4fe01d85825925f3ca4afd0775da3b7ab4b765bedaeffd89dd7ac70db458511"} Jul 06 14:19:32.039988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.039963 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:32.202580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.202546 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle\") pod \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " Jul 06 14:19:32.202754 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.202598 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util\") pod \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " Jul 06 14:19:32.202754 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.202636 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgnhq\" (UniqueName: \"kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq\") pod \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\" (UID: \"6ca87035-0e03-4a45-ab00-bf9cf0006eaf\") " Jul 06 14:19:32.203487 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.203460 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle" (OuterVolumeSpecName: "bundle") pod "6ca87035-0e03-4a45-ab00-bf9cf0006eaf" (UID: "6ca87035-0e03-4a45-ab00-bf9cf0006eaf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:19:32.204828 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.204808 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq" (OuterVolumeSpecName: "kube-api-access-jgnhq") pod "6ca87035-0e03-4a45-ab00-bf9cf0006eaf" (UID: "6ca87035-0e03-4a45-ab00-bf9cf0006eaf"). InnerVolumeSpecName "kube-api-access-jgnhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:19:32.210297 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.210265 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util" (OuterVolumeSpecName: "util") pod "6ca87035-0e03-4a45-ab00-bf9cf0006eaf" (UID: "6ca87035-0e03-4a45-ab00-bf9cf0006eaf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:19:32.303793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.303754 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:32.303793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.303784 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgnhq\" (UniqueName: \"kubernetes.io/projected/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-kube-api-access-jgnhq\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:32.303793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.303795 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ca87035-0e03-4a45-ab00-bf9cf0006eaf-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:19:32.926231 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.926179 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" event={"ID":"6ca87035-0e03-4a45-ab00-bf9cf0006eaf","Type":"ContainerDied","Data":"16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c"} Jul 06 14:19:32.926231 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.926216 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af7bcz8" Jul 06 14:19:32.926438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:32.926265 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16a341f29173c25bacedab86301275c0d0541ab88ec3e49dd642fdce29057f4c" Jul 06 14:19:40.418230 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418184 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb"] Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418507 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="util" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418519 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="util" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418537 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="extract" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418545 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="extract" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418560 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="pull" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418567 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="pull" Jul 06 14:19:40.418609 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.418609 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ca87035-0e03-4a45-ab00-bf9cf0006eaf" containerName="extract" Jul 06 14:19:40.422592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.422577 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.429538 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.429519 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-98nm6\"" Jul 06 14:19:40.430063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.430048 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 14:19:40.431173 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.431158 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 14:19:40.432539 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.432525 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 14:19:40.434482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.434468 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 14:19:40.494645 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.494610 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb"] Jul 06 14:19:40.561289 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.561260 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-tmp\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.561442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.561309 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfmj\" (UniqueName: \"kubernetes.io/projected/709e5301-fad5-4186-884d-246d61b4014a-kube-api-access-htfmj\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.561442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.561333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.561442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.561393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.561562 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.561503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-manifests\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.662832 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.662778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-manifests\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.662832 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.662839 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-tmp\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.663095 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.662869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfmj\" (UniqueName: \"kubernetes.io/projected/709e5301-fad5-4186-884d-246d61b4014a-kube-api-access-htfmj\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.663095 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.662888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.663095 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.662909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.663308 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.663284 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-manifests\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.665303 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.665274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/709e5301-fad5-4186-884d-246d61b4014a-tmp\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.665544 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.665524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.665605 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.665552 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/709e5301-fad5-4186-884d-246d61b4014a-webhook-cert\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.680804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.680739 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfmj\" (UniqueName: \"kubernetes.io/projected/709e5301-fad5-4186-884d-246d61b4014a-kube-api-access-htfmj\") pod \"opendatahub-operator-controller-manager-84ddbc8dff-86ljb\" (UID: \"709e5301-fad5-4186-884d-246d61b4014a\") " pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.731793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.731757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:40.880384 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.880358 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb"] Jul 06 14:19:40.881330 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:19:40.881304 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709e5301_fad5_4186_884d_246d61b4014a.slice/crio-2892cc605a257f56f7ef45fc989e4ccbb15ae0e0a3358527de74e559bf40b50a WatchSource:0}: Error finding container 2892cc605a257f56f7ef45fc989e4ccbb15ae0e0a3358527de74e559bf40b50a: Status 404 returned error can't find the container with id 2892cc605a257f56f7ef45fc989e4ccbb15ae0e0a3358527de74e559bf40b50a Jul 06 14:19:40.961272 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:40.961239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" event={"ID":"709e5301-fad5-4186-884d-246d61b4014a","Type":"ContainerStarted","Data":"2892cc605a257f56f7ef45fc989e4ccbb15ae0e0a3358527de74e559bf40b50a"} Jul 06 14:19:44.651619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:44.651587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:19:44.655158 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:44.655133 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:19:44.662040 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:44.662021 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 14:19:44.983731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:44.983684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" event={"ID":"709e5301-fad5-4186-884d-246d61b4014a","Type":"ContainerStarted","Data":"aebde5632609b7f1ae86c7724ddda1e6e3aea0fdcbe69fdae145a6c64dfd7fff"} Jul 06 14:19:45.989554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:45.989519 2581 generic.go:356] "Generic (PLEG): container finished" podID="709e5301-fad5-4186-884d-246d61b4014a" containerID="aebde5632609b7f1ae86c7724ddda1e6e3aea0fdcbe69fdae145a6c64dfd7fff" exitCode=0 Jul 06 14:19:45.989950 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:45.989603 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" event={"ID":"709e5301-fad5-4186-884d-246d61b4014a","Type":"ContainerDied","Data":"aebde5632609b7f1ae86c7724ddda1e6e3aea0fdcbe69fdae145a6c64dfd7fff"} Jul 06 14:19:45.990071 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:45.990056 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:19:48.000725 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:48.000692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" event={"ID":"709e5301-fad5-4186-884d-246d61b4014a","Type":"ContainerStarted","Data":"0382e12241c37f6560d971d430f28ba903c966408d69288f68d7154281655ff3"} Jul 06 14:19:48.001256 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:48.000842 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:19:48.061568 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:48.061514 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" podStartSLOduration=2.933248105 podStartE2EDuration="8.061500068s" totalImagesPullingTime="4.085870231s" totalInitContainerRuntime="1.042381732s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:19:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:19:48.061146798 +0000 UTC m=+303.755987045" watchObservedRunningTime="2026-07-06 14:19:48.061500068 +0000 UTC m=+303.756340314" Jul 06 14:19:59.008128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:19:59.008092 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84ddbc8dff-86ljb" Jul 06 14:20:01.345773 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.345739 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2"] Jul 06 14:20:01.351655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.351631 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.357756 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.357715 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 14:20:01.358276 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.358258 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 14:20:01.358849 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.358835 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 14:20:01.359675 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.359660 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 14:20:01.416884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.416845 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.417054 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.416929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tmp\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.417054 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.416960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2m8kq\" (UniqueName: \"kubernetes.io/projected/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-kube-api-access-2m8kq\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.454851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.454813 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2"] Jul 06 14:20:01.518177 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.518143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tmp\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.518379 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.518213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2m8kq\" (UniqueName: \"kubernetes.io/projected/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-kube-api-access-2m8kq\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.518379 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.518237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.518379 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:20:01.518349 2581 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 06 14:20:01.518489 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:20:01.518421 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs podName:af9030c5-d31d-4e00-82e6-c29a0a8bdd73 nodeName:}" failed. No retries permitted until 2026-07-06 14:20:02.018404012 +0000 UTC m=+317.713244237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs") pod "kube-auth-proxy-6fbc98d46f-zzgm2" (UID: "af9030c5-d31d-4e00-82e6-c29a0a8bdd73") : secret "kube-auth-proxy-tls" not found Jul 06 14:20:01.520494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.520475 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tmp\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:01.558384 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:01.558348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2m8kq\" (UniqueName: \"kubernetes.io/projected/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-kube-api-access-2m8kq\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:02.020945 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.020909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:02.023471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.023445 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af9030c5-d31d-4e00-82e6-c29a0a8bdd73-tls-certs\") pod \"kube-auth-proxy-6fbc98d46f-zzgm2\" (UID: \"af9030c5-d31d-4e00-82e6-c29a0a8bdd73\") " pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:02.260788 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.260755 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" Jul 06 14:20:02.396038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.395991 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mh8s"] Jul 06 14:20:02.398898 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.398878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:02.405116 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.405098 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 14:20:02.406536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.406521 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gb97s\"" Jul 06 14:20:02.423563 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.423544 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9h5k\" (UniqueName: \"kubernetes.io/projected/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-kube-api-access-p9h5k\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:02.423664 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.423583 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:02.441585 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:02.441559 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf9030c5_d31d_4e00_82e6_c29a0a8bdd73.slice/crio-41fe7eb0ad06d085e6f80d93e487000f9ef5d18a5bd4f41f1f7d76862f9d6de0 WatchSource:0}: Error finding container 41fe7eb0ad06d085e6f80d93e487000f9ef5d18a5bd4f41f1f7d76862f9d6de0: Status 404 returned error can't find the container with id 41fe7eb0ad06d085e6f80d93e487000f9ef5d18a5bd4f41f1f7d76862f9d6de0 Jul 06 14:20:02.446212 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.446169 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mh8s"] Jul 06 14:20:02.447396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.447378 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2"] Jul 06 14:20:02.524747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.524717 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9h5k\" (UniqueName: \"kubernetes.io/projected/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-kube-api-access-p9h5k\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:02.524940 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.524771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:02.524940 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:20:02.524869 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 14:20:02.524940 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:20:02.524927 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert podName:7a2dac3a-f47b-48dc-a22e-8329cea33e9d nodeName:}" failed. No retries permitted until 2026-07-06 14:20:03.024909612 +0000 UTC m=+318.719749838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert") pod "odh-model-controller-7998c67dc4-7mh8s" (UID: "7a2dac3a-f47b-48dc-a22e-8329cea33e9d") : secret "odh-model-controller-webhook-cert" not found Jul 06 14:20:02.545690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:02.545660 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9h5k\" (UniqueName: \"kubernetes.io/projected/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-kube-api-access-p9h5k\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:03.028625 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:03.028589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:03.031276 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:03.031252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a2dac3a-f47b-48dc-a22e-8329cea33e9d-cert\") pod \"odh-model-controller-7998c67dc4-7mh8s\" (UID: \"7a2dac3a-f47b-48dc-a22e-8329cea33e9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:03.063696 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:03.063659 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" event={"ID":"af9030c5-d31d-4e00-82e6-c29a0a8bdd73","Type":"ContainerStarted","Data":"41fe7eb0ad06d085e6f80d93e487000f9ef5d18a5bd4f41f1f7d76862f9d6de0"} Jul 06 14:20:03.309677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:03.309580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:03.464487 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:03.464353 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-7mh8s"] Jul 06 14:20:03.466955 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:03.466928 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a2dac3a_f47b_48dc_a22e_8329cea33e9d.slice/crio-cc4d9253223e12ac3e126b24abeffc14fd074dfd0f3ad43312d6844b528a8fce WatchSource:0}: Error finding container cc4d9253223e12ac3e126b24abeffc14fd074dfd0f3ad43312d6844b528a8fce: Status 404 returned error can't find the container with id cc4d9253223e12ac3e126b24abeffc14fd074dfd0f3ad43312d6844b528a8fce Jul 06 14:20:04.069632 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.069597 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" event={"ID":"7a2dac3a-f47b-48dc-a22e-8329cea33e9d","Type":"ContainerStarted","Data":"cc4d9253223e12ac3e126b24abeffc14fd074dfd0f3ad43312d6844b528a8fce"} Jul 06 14:20:04.278122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.278087 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9z8hp"] Jul 06 14:20:04.280395 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.280368 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.291961 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.291932 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 14:20:04.297608 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.297584 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9z8hp"] Jul 06 14:20:04.302152 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.302124 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rqnbz\"" Jul 06 14:20:04.340746 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.340668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pm4m\" (UniqueName: \"kubernetes.io/projected/59f9a3dc-92b9-4d63-b337-92521815c82a-kube-api-access-4pm4m\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.340746 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.340713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59f9a3dc-92b9-4d63-b337-92521815c82a-tls-certs\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.441590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.441553 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pm4m\" (UniqueName: \"kubernetes.io/projected/59f9a3dc-92b9-4d63-b337-92521815c82a-kube-api-access-4pm4m\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.441790 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.441614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59f9a3dc-92b9-4d63-b337-92521815c82a-tls-certs\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.444156 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.444133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59f9a3dc-92b9-4d63-b337-92521815c82a-tls-certs\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.469988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.469961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pm4m\" (UniqueName: \"kubernetes.io/projected/59f9a3dc-92b9-4d63-b337-92521815c82a-kube-api-access-4pm4m\") pod \"model-serving-api-54d89bd5fc-9z8hp\" (UID: \"59f9a3dc-92b9-4d63-b337-92521815c82a\") " pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.592022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.591922 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:04.786137 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:04.786031 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9z8hp"] Jul 06 14:20:04.789425 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:04.789398 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f9a3dc_92b9_4d63_b337_92521815c82a.slice/crio-8f20c01c90ec7b56c481ec9d31117a1a25919127845de329d5c375217b807776 WatchSource:0}: Error finding container 8f20c01c90ec7b56c481ec9d31117a1a25919127845de329d5c375217b807776: Status 404 returned error can't find the container with id 8f20c01c90ec7b56c481ec9d31117a1a25919127845de329d5c375217b807776 Jul 06 14:20:05.075463 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.075423 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" event={"ID":"59f9a3dc-92b9-4d63-b337-92521815c82a","Type":"ContainerStarted","Data":"8f20c01c90ec7b56c481ec9d31117a1a25919127845de329d5c375217b807776"} Jul 06 14:20:05.491979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.491085 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:05.495710 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.495684 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.500930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.500455 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 14:20:05.500930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.500462 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 14:20:05.500930 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.500784 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9qqq5\"" Jul 06 14:20:05.501437 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.501261 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 14:20:05.501437 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.501280 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 14:20:05.522874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.522845 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:05.552243 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552345 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mcjg\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.552629 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.552489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mcjg\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653232 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653355 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.653569 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653393 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.654004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.653978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.656614 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.656587 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.657374 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.657349 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.658599 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.658575 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.658980 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.658944 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.668629 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.668577 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.669990 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.669964 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mcjg\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg\") pod \"istiod-openshift-gateway-7574459656-d4wtv\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:05.808919 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:05.808837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:06.346511 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:06.346479 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:06.349408 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:06.349378 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3458a416_25ec_4401_9da5_e8caadf85a90.slice/crio-6473a0144b13d3e18168a43b11e8c390095f35dcb0f2255f5993b7bd2f3dc485 WatchSource:0}: Error finding container 6473a0144b13d3e18168a43b11e8c390095f35dcb0f2255f5993b7bd2f3dc485: Status 404 returned error can't find the container with id 6473a0144b13d3e18168a43b11e8c390095f35dcb0f2255f5993b7bd2f3dc485 Jul 06 14:20:07.088838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:07.088800 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" event={"ID":"3458a416-25ec-4401-9da5-e8caadf85a90","Type":"ContainerStarted","Data":"6473a0144b13d3e18168a43b11e8c390095f35dcb0f2255f5993b7bd2f3dc485"} Jul 06 14:20:08.096443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.096404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" event={"ID":"59f9a3dc-92b9-4d63-b337-92521815c82a","Type":"ContainerStarted","Data":"2b64c901c26c593c88b21b0d43b443fd6195ecccf0ec4b93d93904b7b424a5dd"} Jul 06 14:20:08.096933 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.096528 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:08.098782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.098746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" event={"ID":"af9030c5-d31d-4e00-82e6-c29a0a8bdd73","Type":"ContainerStarted","Data":"a753e4afff768a5eeb4a04d90a5d9f513fe9b9d8a1d1f43c1e3b89516a32714a"} Jul 06 14:20:08.100640 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.100616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" event={"ID":"7a2dac3a-f47b-48dc-a22e-8329cea33e9d","Type":"ContainerStarted","Data":"9d0c20e49b5ef64fb4aa6cacbbb282db27786e7bf8742dfb7ca496d1136da5dc"} Jul 06 14:20:08.100979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.100963 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:08.178355 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.178298 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" podStartSLOduration=0.964751296 podStartE2EDuration="4.178282175s" totalImagesPullingTime="3.213530879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:08.170824952 +0000 UTC m=+323.865665200" watchObservedRunningTime="2026-07-06 14:20:08.178282175 +0000 UTC m=+323.873122400" Jul 06 14:20:08.205332 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.205281 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" podStartSLOduration=1.67158695 podStartE2EDuration="6.205263756s" totalImagesPullingTime="4.533676806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:08.203229674 +0000 UTC m=+323.898069917" watchObservedRunningTime="2026-07-06 14:20:08.205263756 +0000 UTC m=+323.900103981" Jul 06 14:20:08.233775 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:08.233722 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fbc98d46f-zzgm2" podStartSLOduration=1.668582263 podStartE2EDuration="7.233706998s" totalImagesPullingTime="5.565124735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:08.232322515 +0000 UTC m=+323.927162760" watchObservedRunningTime="2026-07-06 14:20:08.233706998 +0000 UTC m=+323.928547245" Jul 06 14:20:15.256122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.256087 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8svfr"] Jul 06 14:20:15.267070 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.267041 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.274070 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.274041 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8"] Jul 06 14:20:15.274637 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.274616 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mm8k6\"" Jul 06 14:20:15.276142 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.276082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 14:20:15.277241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.277223 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.285525 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.285503 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 14:20:15.285986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.285802 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-q88rc\"" Jul 06 14:20:15.299946 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.299922 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8svfr"] Jul 06 14:20:15.306009 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.305789 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8"] Jul 06 14:20:15.331886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.331855 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46ntm\" (UniqueName: \"kubernetes.io/projected/89e10ed2-7069-4390-add0-19d9a7a6026f-kube-api-access-46ntm\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.331886 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.331893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbf0d4e2-eb60-443d-8251-a82e632625dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.332113 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.331921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgd28\" (UniqueName: \"kubernetes.io/projected/dbf0d4e2-eb60-443d-8251-a82e632625dd-kube-api-access-sgd28\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.332113 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.332021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89e10ed2-7069-4390-add0-19d9a7a6026f-cert\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.433018 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.432981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46ntm\" (UniqueName: \"kubernetes.io/projected/89e10ed2-7069-4390-add0-19d9a7a6026f-kube-api-access-46ntm\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.433018 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.433022 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbf0d4e2-eb60-443d-8251-a82e632625dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.433287 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.433049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgd28\" (UniqueName: \"kubernetes.io/projected/dbf0d4e2-eb60-443d-8251-a82e632625dd-kube-api-access-sgd28\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.433287 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.433106 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89e10ed2-7069-4390-add0-19d9a7a6026f-cert\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.435985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.435957 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dbf0d4e2-eb60-443d-8251-a82e632625dd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.436135 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.436072 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/89e10ed2-7069-4390-add0-19d9a7a6026f-cert\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.447427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.447394 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46ntm\" (UniqueName: \"kubernetes.io/projected/89e10ed2-7069-4390-add0-19d9a7a6026f-kube-api-access-46ntm\") pod \"kserve-controller-manager-d6f4b65b8-8svfr\" (UID: \"89e10ed2-7069-4390-add0-19d9a7a6026f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.447769 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.447744 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgd28\" (UniqueName: \"kubernetes.io/projected/dbf0d4e2-eb60-443d-8251-a82e632625dd-kube-api-access-sgd28\") pod \"llmisvc-controller-manager-777c6bd7d5-xppf8\" (UID: \"dbf0d4e2-eb60-443d-8251-a82e632625dd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.581163 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.581068 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:15.591255 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.591220 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:15.792128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.792089 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-8svfr"] Jul 06 14:20:15.798659 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:15.798630 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e10ed2_7069_4390_add0_19d9a7a6026f.slice/crio-e3b2099cdc39f5de3be3cb1127ab209dd99c50562f6251dcc4f8f289d850b2ea WatchSource:0}: Error finding container e3b2099cdc39f5de3be3cb1127ab209dd99c50562f6251dcc4f8f289d850b2ea: Status 404 returned error can't find the container with id e3b2099cdc39f5de3be3cb1127ab209dd99c50562f6251dcc4f8f289d850b2ea Jul 06 14:20:15.809978 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:15.809950 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8"] Jul 06 14:20:15.810908 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:15.810884 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddbf0d4e2_eb60_443d_8251_a82e632625dd.slice/crio-dff77fec962324c7bf51b51ad40342108fd40cd134115c4082e317ba161f2a47 WatchSource:0}: Error finding container dff77fec962324c7bf51b51ad40342108fd40cd134115c4082e317ba161f2a47: Status 404 returned error can't find the container with id dff77fec962324c7bf51b51ad40342108fd40cd134115c4082e317ba161f2a47 Jul 06 14:20:16.138370 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:16.138336 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" event={"ID":"dbf0d4e2-eb60-443d-8251-a82e632625dd","Type":"ContainerStarted","Data":"dff77fec962324c7bf51b51ad40342108fd40cd134115c4082e317ba161f2a47"} Jul 06 14:20:16.139494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:16.139468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" event={"ID":"89e10ed2-7069-4390-add0-19d9a7a6026f","Type":"ContainerStarted","Data":"e3b2099cdc39f5de3be3cb1127ab209dd99c50562f6251dcc4f8f289d850b2ea"} Jul 06 14:20:18.426266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.426234 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2"] Jul 06 14:20:18.430670 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.430650 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.459923 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.459893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.459936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.459967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.459994 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.460036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.460097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbw7t\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-kube-api-access-mbw7t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.460159 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.460241 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.460450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.460274 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.480098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.480070 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-mn59g\"" Jul 06 14:20:18.561027 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.560997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbw7t\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-kube-api-access-mbw7t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561047 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561109 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561172 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561507 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561507 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.561507 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.562217 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.561997 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.562447 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.562405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.562530 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.562451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.562575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.562533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.562952 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.562927 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.564139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.564116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.565107 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.565087 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.758822 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.758756 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbw7t\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-kube-api-access-mbw7t\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:18.790698 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.790613 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2"] Jul 06 14:20:18.798702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:18.798674 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb9dfc8d-63f5-4f66-9c7a-21d451079b30-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48llv2\" (UID: \"eb9dfc8d-63f5-4f66-9c7a-21d451079b30\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:19.041862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:19.041834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:19.109829 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:19.109743 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9z8hp" Jul 06 14:20:19.155035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:19.154994 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" event={"ID":"89e10ed2-7069-4390-add0-19d9a7a6026f","Type":"ContainerStarted","Data":"27d8cf15ec67785e26226f7fc80ce3582538856f78a2d33768924b4b4c578a53"} Jul 06 14:20:19.155230 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:19.155062 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:20.111729 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:20.111697 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-7mh8s" Jul 06 14:20:20.139210 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:20.139138 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" podStartSLOduration=1.9470820340000001 podStartE2EDuration="5.139116673s" totalImagesPullingTime="3.192034639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:19.347927954 +0000 UTC m=+335.042768211" watchObservedRunningTime="2026-07-06 14:20:20.139116673 +0000 UTC m=+335.833956922" Jul 06 14:20:20.501157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:20.501118 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:20:20.501269 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:20.501204 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:20:20.660509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:20.660478 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2"] Jul 06 14:20:20.663281 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:20:20.663261 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9dfc8d_63f5_4f66_9c7a_21d451079b30.slice/crio-47a9cf0aea337b1c56b7a28300c2da9183f8f6e1919786bcf54cbe8ba0248aa1 WatchSource:0}: Error finding container 47a9cf0aea337b1c56b7a28300c2da9183f8f6e1919786bcf54cbe8ba0248aa1: Status 404 returned error can't find the container with id 47a9cf0aea337b1c56b7a28300c2da9183f8f6e1919786bcf54cbe8ba0248aa1 Jul 06 14:20:21.174277 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.174235 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" event={"ID":"eb9dfc8d-63f5-4f66-9c7a-21d451079b30","Type":"ContainerStarted","Data":"47a9cf0aea337b1c56b7a28300c2da9183f8f6e1919786bcf54cbe8ba0248aa1"} Jul 06 14:20:21.175960 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.175927 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" event={"ID":"3458a416-25ec-4401-9da5-e8caadf85a90","Type":"ContainerStarted","Data":"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa"} Jul 06 14:20:21.176146 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.176125 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:21.177642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.177610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" event={"ID":"dbf0d4e2-eb60-443d-8251-a82e632625dd","Type":"ContainerStarted","Data":"3235fb792904a8d9d9f5881b18915c791e14189dd64be656a0bff6f0e46f4cc9"} Jul 06 14:20:21.177846 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.177825 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:20:21.177962 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.177887 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-d4wtv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:20:21.178026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.177982 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:20:21.414875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.414822 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" podStartSLOduration=2.266168533 podStartE2EDuration="16.414802316s" totalImagesPullingTime="14.148633783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:21.328899942 +0000 UTC m=+337.023740201" watchObservedRunningTime="2026-07-06 14:20:21.414802316 +0000 UTC m=+337.109642562" Jul 06 14:20:21.415600 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.415568 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" podStartSLOduration=1.732455065 podStartE2EDuration="6.415556884s" totalImagesPullingTime="4.683101819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:21.413968495 +0000 UTC m=+337.108808809" watchObservedRunningTime="2026-07-06 14:20:21.415556884 +0000 UTC m=+337.110397131" Jul 06 14:20:21.530435 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:21.530398 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:22.183062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:22.183032 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:23.185936 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.185892 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" containerName="discovery" containerID="cri-o://b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa" gracePeriod=30 Jul 06 14:20:23.228282 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.228247 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:20:23.228355 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.228324 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:20:23.228393 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.228353 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:20:23.444387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.444365 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:23.506534 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506499 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506561 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mcjg\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506590 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506640 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506675 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506714 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.506904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.506739 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token\") pod \"3458a416-25ec-4401-9da5-e8caadf85a90\" (UID: \"3458a416-25ec-4401-9da5-e8caadf85a90\") " Jul 06 14:20:23.507576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.507541 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:20:23.510105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.510077 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts" (OuterVolumeSpecName: "cacerts") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:20:23.510448 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.510414 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs" (OuterVolumeSpecName: "local-certs") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:20:23.510552 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.510538 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:20:23.510552 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.510534 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg" (OuterVolumeSpecName: "kube-api-access-7mcjg") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "kube-api-access-7mcjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:20:23.510647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.510561 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:20:23.512146 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.512112 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token" (OuterVolumeSpecName: "istio-token") pod "3458a416-25ec-4401-9da5-e8caadf85a90" (UID: "3458a416-25ec-4401-9da5-e8caadf85a90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:20:23.607948 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607906 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mcjg\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-kube-api-access-7mcjg\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.607948 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607941 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-kubeconfig\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.607948 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607951 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-cacerts\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.607948 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607958 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3458a416-25ec-4401-9da5-e8caadf85a90-local-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.608238 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607966 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-ca-configmap\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.608238 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607974 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3458a416-25ec-4401-9da5-e8caadf85a90-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:23.608238 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:23.607982 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3458a416-25ec-4401-9da5-e8caadf85a90-istio-csr-dns-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:20:24.191415 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.191379 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" event={"ID":"eb9dfc8d-63f5-4f66-9c7a-21d451079b30","Type":"ContainerStarted","Data":"d5ecfa350521ca331e728da8541cec97797d824a90bd9d1268ea420a5c790888"} Jul 06 14:20:24.192783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.192756 2581 generic.go:356] "Generic (PLEG): container finished" podID="3458a416-25ec-4401-9da5-e8caadf85a90" containerID="b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa" exitCode=0 Jul 06 14:20:24.192896 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.192800 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" Jul 06 14:20:24.192896 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.192845 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" event={"ID":"3458a416-25ec-4401-9da5-e8caadf85a90","Type":"ContainerDied","Data":"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa"} Jul 06 14:20:24.192896 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.192887 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv" event={"ID":"3458a416-25ec-4401-9da5-e8caadf85a90","Type":"ContainerDied","Data":"6473a0144b13d3e18168a43b11e8c390095f35dcb0f2255f5993b7bd2f3dc485"} Jul 06 14:20:24.192992 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.192905 2581 scope.go:117] "RemoveContainer" containerID="b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa" Jul 06 14:20:24.202400 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.202382 2581 scope.go:117] "RemoveContainer" containerID="b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa" Jul 06 14:20:24.202665 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:20:24.202648 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa\": container with ID starting with b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa not found: ID does not exist" containerID="b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa" Jul 06 14:20:24.202727 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.202675 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa"} err="failed to get container status \"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa\": rpc error: code = NotFound desc = could not find container \"b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa\": container with ID starting with b33e7b7371f71fe2da23bc31c52c543363ae5d75fdf9894e942afe3dabe7abaa not found: ID does not exist" Jul 06 14:20:24.224738 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.224688 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" podStartSLOduration=3.661958819 podStartE2EDuration="6.224671788s" totalImagesPullingTime="2.562712969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:20:24.22266146 +0000 UTC m=+339.917501706" watchObservedRunningTime="2026-07-06 14:20:24.224671788 +0000 UTC m=+339.919512035" Jul 06 14:20:24.245929 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.245899 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:24.255534 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.255509 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-d4wtv"] Jul 06 14:20:24.814330 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:24.814287 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" path="/var/lib/kubelet/pods/3458a416-25ec-4401-9da5-e8caadf85a90/volumes" Jul 06 14:20:25.042578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:25.042540 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:25.046831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:25.046808 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:25.198086 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:25.198052 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:25.198985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:25.198961 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48llv2" Jul 06 14:20:50.166188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:50.166159 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-8svfr" Jul 06 14:20:52.184341 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:20:52.184307 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xppf8" Jul 06 14:21:03.014435 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.014403 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:03.014819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.014699 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" containerName="discovery" Jul 06 14:21:03.014819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.014710 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" containerName="discovery" Jul 06 14:21:03.014819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.014765 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3458a416-25ec-4401-9da5-e8caadf85a90" containerName="discovery" Jul 06 14:21:03.025225 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.025180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:03.037935 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.037911 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-x84rp\"" Jul 06 14:21:03.039237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.039218 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 14:21:03.039342 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.039242 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 14:21:03.041485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.041465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdkx\" (UniqueName: \"kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx\") pod \"kuadrant-operator-catalog-c5n98\" (UID: \"180db15e-e6dd-4a07-8dfe-2e4c07538bc9\") " pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:03.053284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.053259 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:03.142813 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.142764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdkx\" (UniqueName: \"kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx\") pod \"kuadrant-operator-catalog-c5n98\" (UID: \"180db15e-e6dd-4a07-8dfe-2e4c07538bc9\") " pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:03.168590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.168557 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdkx\" (UniqueName: \"kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx\") pod \"kuadrant-operator-catalog-c5n98\" (UID: \"180db15e-e6dd-4a07-8dfe-2e4c07538bc9\") " pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:03.252654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.252621 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:03.252879 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.252856 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:03.406830 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.406805 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:03.409457 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:03.409429 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180db15e_e6dd_4a07_8dfe_2e4c07538bc9.slice/crio-fed1e4fd7e0a9cb9311f622ab0c3e6f2bf55abdb9441ebf4c27326f2e3b5e719 WatchSource:0}: Error finding container fed1e4fd7e0a9cb9311f622ab0c3e6f2bf55abdb9441ebf4c27326f2e3b5e719: Status 404 returned error can't find the container with id fed1e4fd7e0a9cb9311f622ab0c3e6f2bf55abdb9441ebf4c27326f2e3b5e719 Jul 06 14:21:03.478938 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.478904 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqdzz"] Jul 06 14:21:03.481835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.481821 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:03.504634 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.504607 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqdzz"] Jul 06 14:21:03.546513 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.546485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjnsz\" (UniqueName: \"kubernetes.io/projected/d9d16544-da2d-4601-b300-0e2f3f11b062-kube-api-access-wjnsz\") pod \"kuadrant-operator-catalog-hqdzz\" (UID: \"d9d16544-da2d-4601-b300-0e2f3f11b062\") " pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:03.647472 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.647384 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjnsz\" (UniqueName: \"kubernetes.io/projected/d9d16544-da2d-4601-b300-0e2f3f11b062-kube-api-access-wjnsz\") pod \"kuadrant-operator-catalog-hqdzz\" (UID: \"d9d16544-da2d-4601-b300-0e2f3f11b062\") " pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:03.663842 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.663809 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjnsz\" (UniqueName: \"kubernetes.io/projected/d9d16544-da2d-4601-b300-0e2f3f11b062-kube-api-access-wjnsz\") pod \"kuadrant-operator-catalog-hqdzz\" (UID: \"d9d16544-da2d-4601-b300-0e2f3f11b062\") " pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:03.796344 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.796302 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:03.953564 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:03.953539 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hqdzz"] Jul 06 14:21:03.955930 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:03.955905 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d16544_da2d_4601_b300_0e2f3f11b062.slice/crio-ff172893d725a7deaf9a89fa775f3d64735c7c8f9661c9acea6ac130fd27a8bd WatchSource:0}: Error finding container ff172893d725a7deaf9a89fa775f3d64735c7c8f9661c9acea6ac130fd27a8bd: Status 404 returned error can't find the container with id ff172893d725a7deaf9a89fa775f3d64735c7c8f9661c9acea6ac130fd27a8bd Jul 06 14:21:04.372845 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:04.372764 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" event={"ID":"d9d16544-da2d-4601-b300-0e2f3f11b062","Type":"ContainerStarted","Data":"ff172893d725a7deaf9a89fa775f3d64735c7c8f9661c9acea6ac130fd27a8bd"} Jul 06 14:21:04.373979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:04.373945 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" event={"ID":"180db15e-e6dd-4a07-8dfe-2e4c07538bc9","Type":"ContainerStarted","Data":"fed1e4fd7e0a9cb9311f622ab0c3e6f2bf55abdb9441ebf4c27326f2e3b5e719"} Jul 06 14:21:06.385035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.385001 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" event={"ID":"d9d16544-da2d-4601-b300-0e2f3f11b062","Type":"ContainerStarted","Data":"6b859f291562150de74f26ecd9d754154b095eb7411320210099c4eec8d96626"} Jul 06 14:21:06.386318 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.386294 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" event={"ID":"180db15e-e6dd-4a07-8dfe-2e4c07538bc9","Type":"ContainerStarted","Data":"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7"} Jul 06 14:21:06.386446 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.386364 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" podUID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" containerName="registry-server" containerID="cri-o://9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7" gracePeriod=2 Jul 06 14:21:06.407798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.407754 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" podStartSLOduration=1.4834035939999999 podStartE2EDuration="3.407739821s" totalImagesPullingTime="1.924336227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:06.406149365 +0000 UTC m=+382.100989613" watchObservedRunningTime="2026-07-06 14:21:06.407739821 +0000 UTC m=+382.102580067" Jul 06 14:21:06.426162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.426110 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" podStartSLOduration=1.961951638 podStartE2EDuration="4.426096504s" totalImagesPullingTime="2.464144866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:06.425313842 +0000 UTC m=+382.120154090" watchObservedRunningTime="2026-07-06 14:21:06.426096504 +0000 UTC m=+382.120936750" Jul 06 14:21:06.628750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.628725 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:06.673469 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.673406 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvdkx\" (UniqueName: \"kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx\") pod \"180db15e-e6dd-4a07-8dfe-2e4c07538bc9\" (UID: \"180db15e-e6dd-4a07-8dfe-2e4c07538bc9\") " Jul 06 14:21:06.675749 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.675721 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx" (OuterVolumeSpecName: "kube-api-access-mvdkx") pod "180db15e-e6dd-4a07-8dfe-2e4c07538bc9" (UID: "180db15e-e6dd-4a07-8dfe-2e4c07538bc9"). InnerVolumeSpecName "kube-api-access-mvdkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:21:06.774098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:06.774068 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvdkx\" (UniqueName: \"kubernetes.io/projected/180db15e-e6dd-4a07-8dfe-2e4c07538bc9-kube-api-access-mvdkx\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:07.392357 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.392319 2581 generic.go:356] "Generic (PLEG): container finished" podID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" containerID="9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7" exitCode=0 Jul 06 14:21:07.392870 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.392378 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" Jul 06 14:21:07.392870 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.392408 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" event={"ID":"180db15e-e6dd-4a07-8dfe-2e4c07538bc9","Type":"ContainerDied","Data":"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7"} Jul 06 14:21:07.392870 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.392498 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c5n98" event={"ID":"180db15e-e6dd-4a07-8dfe-2e4c07538bc9","Type":"ContainerDied","Data":"fed1e4fd7e0a9cb9311f622ab0c3e6f2bf55abdb9441ebf4c27326f2e3b5e719"} Jul 06 14:21:07.392870 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.392541 2581 scope.go:117] "RemoveContainer" containerID="9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7" Jul 06 14:21:07.401858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.401839 2581 scope.go:117] "RemoveContainer" containerID="9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7" Jul 06 14:21:07.402105 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:21:07.402087 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7\": container with ID starting with 9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7 not found: ID does not exist" containerID="9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7" Jul 06 14:21:07.402153 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.402118 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7"} err="failed to get container status \"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7\": rpc error: code = NotFound desc = could not find container \"9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7\": container with ID starting with 9c2aeb420def270378ee13422a202d1d0d25a29d57f56ca700ba04dfe79273f7 not found: ID does not exist" Jul 06 14:21:07.418748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.418725 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:07.425711 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:07.425688 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c5n98"] Jul 06 14:21:08.813683 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:08.813654 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" path="/var/lib/kubelet/pods/180db15e-e6dd-4a07-8dfe-2e4c07538bc9/volumes" Jul 06 14:21:13.796800 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:13.796760 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:13.797217 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:13.796832 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:13.819819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:13.819794 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:14.449311 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:14.449284 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hqdzz" Jul 06 14:21:18.932556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.932523 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l"] Jul 06 14:21:18.933032 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.932864 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" containerName="registry-server" Jul 06 14:21:18.933032 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.932875 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" containerName="registry-server" Jul 06 14:21:18.933032 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.932935 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="180db15e-e6dd-4a07-8dfe-2e4c07538bc9" containerName="registry-server" Jul 06 14:21:18.935965 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.935947 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:18.939227 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.939177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n978r\"" Jul 06 14:21:18.952054 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.952030 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l"] Jul 06 14:21:18.964102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.964073 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:18.964260 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.964107 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:18.964260 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:18.964129 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqtzc\" (UniqueName: \"kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.064957 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.064897 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.065140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.064962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.065140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.064999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqtzc\" (UniqueName: \"kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.065434 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.065408 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.066317 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.065430 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.079241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.079217 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqtzc\" (UniqueName: \"kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.245679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.245647 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:19.330992 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.330956 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98"] Jul 06 14:21:19.334867 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.334841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.347020 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.346952 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98"] Jul 06 14:21:19.367516 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.367486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.367655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.367520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.367655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.367541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdj2\" (UniqueName: \"kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.387267 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.387245 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l"] Jul 06 14:21:19.388745 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:19.388724 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4f9d86_a7b6_4382_8934_667c9d06325c.slice/crio-15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf WatchSource:0}: Error finding container 15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf: Status 404 returned error can't find the container with id 15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf Jul 06 14:21:19.448634 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.448589 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" event={"ID":"de4f9d86-a7b6-4382-8934-667c9d06325c","Type":"ContainerStarted","Data":"15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf"} Jul 06 14:21:19.468632 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.468605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.468750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.468641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.468750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.468669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdj2\" (UniqueName: \"kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.468969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.468949 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.469026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.469005 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.479266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.479240 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdj2\" (UniqueName: \"kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.647731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.647652 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:19.786233 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.786208 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98"] Jul 06 14:21:19.788482 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:19.788458 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e83895_238a_4c5a_88f9_c48b9b1a7a08.slice/crio-3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3 WatchSource:0}: Error finding container 3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3: Status 404 returned error can't find the container with id 3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3 Jul 06 14:21:19.920505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.920475 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c"] Jul 06 14:21:19.923160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.923145 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:19.938600 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.938570 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c"] Jul 06 14:21:19.973465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.973435 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clxvx\" (UniqueName: \"kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:19.973575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.973472 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:19.973575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:19.973549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.073914 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.073879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clxvx\" (UniqueName: \"kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.073914 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.073918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.074105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.073972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.074371 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.074354 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.074434 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.074418 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.085219 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.085162 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clxvx\" (UniqueName: \"kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.251650 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.251619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:20.389835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.389809 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c"] Jul 06 14:21:20.392453 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:20.392425 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969e84d1_1627_4b3e_a0e9_217ad3dbdc53.slice/crio-35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf WatchSource:0}: Error finding container 35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf: Status 404 returned error can't find the container with id 35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf Jul 06 14:21:20.455338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.455313 2581 generic.go:356] "Generic (PLEG): container finished" podID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerID="e6e97f56ca1266ab77a87c31ea4d7d459b473fadb8a0a271439f63eec8edc13c" exitCode=0 Jul 06 14:21:20.455449 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.455390 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" event={"ID":"de4f9d86-a7b6-4382-8934-667c9d06325c","Type":"ContainerDied","Data":"e6e97f56ca1266ab77a87c31ea4d7d459b473fadb8a0a271439f63eec8edc13c"} Jul 06 14:21:20.457094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.457074 2581 generic.go:356] "Generic (PLEG): container finished" podID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerID="e768539411c19aad6d1a77418a1b7321afcbb25bdc5a6f4eed4ba01d8a3c387b" exitCode=0 Jul 06 14:21:20.457182 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.457160 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" event={"ID":"22e83895-238a-4c5a-88f9-c48b9b1a7a08","Type":"ContainerDied","Data":"e768539411c19aad6d1a77418a1b7321afcbb25bdc5a6f4eed4ba01d8a3c387b"} Jul 06 14:21:20.457255 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.457238 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" event={"ID":"22e83895-238a-4c5a-88f9-c48b9b1a7a08","Type":"ContainerStarted","Data":"3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3"} Jul 06 14:21:20.458973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.458946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerStarted","Data":"35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf"} Jul 06 14:21:20.526342 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.526268 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp"] Jul 06 14:21:20.528708 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.528693 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.544442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.544417 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp"] Jul 06 14:21:20.578982 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.578957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.579105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.579001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt4h5\" (UniqueName: \"kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.579105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.579085 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.680180 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.680144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.680359 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.680213 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.680359 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.680248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt4h5\" (UniqueName: \"kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.680544 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.680521 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.680608 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.680589 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.691530 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.691506 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt4h5\" (UniqueName: \"kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:20.864042 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:20.863961 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:21.003323 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.003300 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp"] Jul 06 14:21:21.005645 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:21.005621 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f63ee1_127f_423e_a7a3_389c24bce1ed.slice/crio-de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029 WatchSource:0}: Error finding container de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029: Status 404 returned error can't find the container with id de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029 Jul 06 14:21:21.465786 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.465746 2581 generic.go:356] "Generic (PLEG): container finished" podID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerID="ecd1081dfd73ecb55c28c3b91d0da5652346c99bb086b3667123425e0a9451ac" exitCode=0 Jul 06 14:21:21.465973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.465819 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" event={"ID":"22e83895-238a-4c5a-88f9-c48b9b1a7a08","Type":"ContainerDied","Data":"ecd1081dfd73ecb55c28c3b91d0da5652346c99bb086b3667123425e0a9451ac"} Jul 06 14:21:21.467372 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.467333 2581 generic.go:356] "Generic (PLEG): container finished" podID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerID="eb72bfa81d687a72c1d3ffa04243ab56a4ec36a0119ca19a3b794ad464f08eee" exitCode=0 Jul 06 14:21:21.467436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.467410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerDied","Data":"eb72bfa81d687a72c1d3ffa04243ab56a4ec36a0119ca19a3b794ad464f08eee"} Jul 06 14:21:21.469089 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.469056 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerID="fc931c73464f6d9e01abbde1790a8b13d41fd546bfed89bec57bb86d30dc1ca6" exitCode=0 Jul 06 14:21:21.469234 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.469131 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" event={"ID":"f8f63ee1-127f-423e-a7a3-389c24bce1ed","Type":"ContainerDied","Data":"fc931c73464f6d9e01abbde1790a8b13d41fd546bfed89bec57bb86d30dc1ca6"} Jul 06 14:21:21.469234 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:21.469164 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" event={"ID":"f8f63ee1-127f-423e-a7a3-389c24bce1ed","Type":"ContainerStarted","Data":"de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029"} Jul 06 14:21:22.476171 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.476130 2581 generic.go:356] "Generic (PLEG): container finished" podID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerID="c3d5cb02dfb2fbad281285d81af13e2d69d135cc0ca562e125cdbfa476e9f997" exitCode=0 Jul 06 14:21:22.476489 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.476211 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" event={"ID":"22e83895-238a-4c5a-88f9-c48b9b1a7a08","Type":"ContainerDied","Data":"c3d5cb02dfb2fbad281285d81af13e2d69d135cc0ca562e125cdbfa476e9f997"} Jul 06 14:21:22.478002 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.477978 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerStarted","Data":"4a73c3c04c79871405aca5d1cf60b4dda895a89ce0e183d399ac3cf16aacdde6"} Jul 06 14:21:22.479695 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.479675 2581 generic.go:356] "Generic (PLEG): container finished" podID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerID="1d8b01bd388f79a29e42400a0436e44b4d856388054bb43c5187a9803be615d4" exitCode=0 Jul 06 14:21:22.479786 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.479737 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" event={"ID":"de4f9d86-a7b6-4382-8934-667c9d06325c","Type":"ContainerDied","Data":"1d8b01bd388f79a29e42400a0436e44b4d856388054bb43c5187a9803be615d4"} Jul 06 14:21:22.481610 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.481590 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerID="dfb6da53f7d43a081f7ec7a0a3f403ebc4940ea52b2ebc287785bdeb0902e7f5" exitCode=0 Jul 06 14:21:22.481713 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:22.481629 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" event={"ID":"f8f63ee1-127f-423e-a7a3-389c24bce1ed","Type":"ContainerDied","Data":"dfb6da53f7d43a081f7ec7a0a3f403ebc4940ea52b2ebc287785bdeb0902e7f5"} Jul 06 14:21:23.488471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.488437 2581 generic.go:356] "Generic (PLEG): container finished" podID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerID="4a73c3c04c79871405aca5d1cf60b4dda895a89ce0e183d399ac3cf16aacdde6" exitCode=0 Jul 06 14:21:23.488894 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.488528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerDied","Data":"4a73c3c04c79871405aca5d1cf60b4dda895a89ce0e183d399ac3cf16aacdde6"} Jul 06 14:21:23.490758 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.490735 2581 generic.go:356] "Generic (PLEG): container finished" podID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerID="486f2d48f174295e53c2060e61e5fe5a403a35ecfd0900391d48193cf0023467" exitCode=0 Jul 06 14:21:23.490895 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.490813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" event={"ID":"de4f9d86-a7b6-4382-8934-667c9d06325c","Type":"ContainerDied","Data":"486f2d48f174295e53c2060e61e5fe5a403a35ecfd0900391d48193cf0023467"} Jul 06 14:21:23.492725 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.492706 2581 generic.go:356] "Generic (PLEG): container finished" podID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerID="57b527995dc1874f5d2c1f10cdbd0ffd92dd2222ba84f09bc1dec1809c779cbe" exitCode=0 Jul 06 14:21:23.492810 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.492766 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" event={"ID":"f8f63ee1-127f-423e-a7a3-389c24bce1ed","Type":"ContainerDied","Data":"57b527995dc1874f5d2c1f10cdbd0ffd92dd2222ba84f09bc1dec1809c779cbe"} Jul 06 14:21:23.629398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.629373 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:23.706692 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.706670 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle\") pod \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " Jul 06 14:21:23.706860 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.706723 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdj2\" (UniqueName: \"kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2\") pod \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " Jul 06 14:21:23.706860 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.706744 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util\") pod \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\" (UID: \"22e83895-238a-4c5a-88f9-c48b9b1a7a08\") " Jul 06 14:21:23.707171 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.707139 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle" (OuterVolumeSpecName: "bundle") pod "22e83895-238a-4c5a-88f9-c48b9b1a7a08" (UID: "22e83895-238a-4c5a-88f9-c48b9b1a7a08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:23.708810 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.708789 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2" (OuterVolumeSpecName: "kube-api-access-6jdj2") pod "22e83895-238a-4c5a-88f9-c48b9b1a7a08" (UID: "22e83895-238a-4c5a-88f9-c48b9b1a7a08"). InnerVolumeSpecName "kube-api-access-6jdj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:21:23.712170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.712148 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util" (OuterVolumeSpecName: "util") pod "22e83895-238a-4c5a-88f9-c48b9b1a7a08" (UID: "22e83895-238a-4c5a-88f9-c48b9b1a7a08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:23.808282 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.808253 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:23.808427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.808336 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdj2\" (UniqueName: \"kubernetes.io/projected/22e83895-238a-4c5a-88f9-c48b9b1a7a08-kube-api-access-6jdj2\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:23.808427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:23.808358 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/22e83895-238a-4c5a-88f9-c48b9b1a7a08-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.499428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.499401 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" Jul 06 14:21:24.499864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.499397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98" event={"ID":"22e83895-238a-4c5a-88f9-c48b9b1a7a08","Type":"ContainerDied","Data":"3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3"} Jul 06 14:21:24.499864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.499516 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a4571ce8ca46216f07f92f8f210839547b33a2e65b658e159843bb0a09691c3" Jul 06 14:21:24.501505 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.501479 2581 generic.go:356] "Generic (PLEG): container finished" podID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerID="bce8b0cea551d0d27dcc9641cf88ba4c4d375e8ffaa17aeba45998855fbf758a" exitCode=0 Jul 06 14:21:24.501635 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.501552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerDied","Data":"bce8b0cea551d0d27dcc9641cf88ba4c4d375e8ffaa17aeba45998855fbf758a"} Jul 06 14:21:24.637335 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.637308 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:24.667969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.667945 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:24.715804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715773 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt4h5\" (UniqueName: \"kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5\") pod \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " Jul 06 14:21:24.715975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util\") pod \"de4f9d86-a7b6-4382-8934-667c9d06325c\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " Jul 06 14:21:24.715975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715872 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqtzc\" (UniqueName: \"kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc\") pod \"de4f9d86-a7b6-4382-8934-667c9d06325c\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " Jul 06 14:21:24.715975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715899 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util\") pod \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " Jul 06 14:21:24.715975 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715956 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle\") pod \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\" (UID: \"f8f63ee1-127f-423e-a7a3-389c24bce1ed\") " Jul 06 14:21:24.716169 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.715996 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle\") pod \"de4f9d86-a7b6-4382-8934-667c9d06325c\" (UID: \"de4f9d86-a7b6-4382-8934-667c9d06325c\") " Jul 06 14:21:24.716520 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.716493 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle" (OuterVolumeSpecName: "bundle") pod "f8f63ee1-127f-423e-a7a3-389c24bce1ed" (UID: "f8f63ee1-127f-423e-a7a3-389c24bce1ed"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:24.716828 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.716803 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle" (OuterVolumeSpecName: "bundle") pod "de4f9d86-a7b6-4382-8934-667c9d06325c" (UID: "de4f9d86-a7b6-4382-8934-667c9d06325c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:24.718581 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.718557 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5" (OuterVolumeSpecName: "kube-api-access-dt4h5") pod "f8f63ee1-127f-423e-a7a3-389c24bce1ed" (UID: "f8f63ee1-127f-423e-a7a3-389c24bce1ed"). InnerVolumeSpecName "kube-api-access-dt4h5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:21:24.718666 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.718574 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc" (OuterVolumeSpecName: "kube-api-access-wqtzc") pod "de4f9d86-a7b6-4382-8934-667c9d06325c" (UID: "de4f9d86-a7b6-4382-8934-667c9d06325c"). InnerVolumeSpecName "kube-api-access-wqtzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:21:24.722619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.722596 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util" (OuterVolumeSpecName: "util") pod "f8f63ee1-127f-423e-a7a3-389c24bce1ed" (UID: "f8f63ee1-127f-423e-a7a3-389c24bce1ed"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:24.805971 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.805889 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util" (OuterVolumeSpecName: "util") pod "de4f9d86-a7b6-4382-8934-667c9d06325c" (UID: "de4f9d86-a7b6-4382-8934-667c9d06325c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:24.820241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820181 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt4h5\" (UniqueName: \"kubernetes.io/projected/f8f63ee1-127f-423e-a7a3-389c24bce1ed-kube-api-access-dt4h5\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.820241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820243 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.820241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820261 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqtzc\" (UniqueName: \"kubernetes.io/projected/de4f9d86-a7b6-4382-8934-667c9d06325c-kube-api-access-wqtzc\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.820542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820274 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.820542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820288 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8f63ee1-127f-423e-a7a3-389c24bce1ed-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:24.820542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:24.820300 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de4f9d86-a7b6-4382-8934-667c9d06325c-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:25.507982 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.507944 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" event={"ID":"de4f9d86-a7b6-4382-8934-667c9d06325c","Type":"ContainerDied","Data":"15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf"} Jul 06 14:21:25.508481 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.507999 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15a1f62995f5b1edbefc4f414064fccc31e64e1dd1d3a567a235076432184ddf" Jul 06 14:21:25.508481 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.507954 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l" Jul 06 14:21:25.510032 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.510008 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" Jul 06 14:21:25.510158 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.510002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp" event={"ID":"f8f63ee1-127f-423e-a7a3-389c24bce1ed","Type":"ContainerDied","Data":"de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029"} Jul 06 14:21:25.510158 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.510070 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de0352cde2cf5f88526a9def51cf3e8508874256f1165fba43c64cb20e398029" Jul 06 14:21:25.646758 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.646726 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:25.727144 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.727117 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util\") pod \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " Jul 06 14:21:25.727362 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.727181 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clxvx\" (UniqueName: \"kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx\") pod \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " Jul 06 14:21:25.727362 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.727235 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle\") pod \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\" (UID: \"969e84d1-1627-4b3e-a0e9-217ad3dbdc53\") " Jul 06 14:21:25.727805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.727772 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle" (OuterVolumeSpecName: "bundle") pod "969e84d1-1627-4b3e-a0e9-217ad3dbdc53" (UID: "969e84d1-1627-4b3e-a0e9-217ad3dbdc53"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:25.729353 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.729323 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx" (OuterVolumeSpecName: "kube-api-access-clxvx") pod "969e84d1-1627-4b3e-a0e9-217ad3dbdc53" (UID: "969e84d1-1627-4b3e-a0e9-217ad3dbdc53"). InnerVolumeSpecName "kube-api-access-clxvx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:21:25.827809 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.827725 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util" (OuterVolumeSpecName: "util") pod "969e84d1-1627-4b3e-a0e9-217ad3dbdc53" (UID: "969e84d1-1627-4b3e-a0e9-217ad3dbdc53"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:21:25.828075 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.828062 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clxvx\" (UniqueName: \"kubernetes.io/projected/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-kube-api-access-clxvx\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:25.828125 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.828081 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:25.828125 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:25.828095 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/969e84d1-1627-4b3e-a0e9-217ad3dbdc53-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:21:26.517016 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:26.516989 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" Jul 06 14:21:26.517016 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:26.516997 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c" event={"ID":"969e84d1-1627-4b3e-a0e9-217ad3dbdc53","Type":"ContainerDied","Data":"35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf"} Jul 06 14:21:26.517439 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:26.517047 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35d973ae294abef6824c6b4a58269e602a064926227ff58a4533efe194929bbf" Jul 06 14:21:36.507202 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507165 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx"] Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507516 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507529 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507540 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507545 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507552 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507562 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507571 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507576 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507583 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507587 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507594 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507599 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="pull" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507608 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507615 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507620 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507624 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="extract" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507633 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="util" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507641 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="util" Jul 06 14:21:36.507643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507650 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507656 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507661 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507666 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507675 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507680 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="util" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507731 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f63ee1-127f-423e-a7a3-389c24bce1ed" containerName="extract" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507741 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="969e84d1-1627-4b3e-a0e9-217ad3dbdc53" containerName="extract" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507747 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e83895-238a-4c5a-88f9-c48b9b1a7a08" containerName="extract" Jul 06 14:21:36.508214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.507755 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4f9d86-a7b6-4382-8934-667c9d06325c" containerName="extract" Jul 06 14:21:36.510553 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.510534 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:36.516810 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.516787 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kbgh9\"" Jul 06 14:21:36.529421 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.529391 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx"] Jul 06 14:21:36.618868 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.618836 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbw4s\" (UniqueName: \"kubernetes.io/projected/fd79a6ce-4f90-49ed-8808-64a3f8e82398-kube-api-access-nbw4s\") pod \"authorino-operator-75cc8d5cb7-6jxtx\" (UID: \"fd79a6ce-4f90-49ed-8808-64a3f8e82398\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:36.719976 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.719940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbw4s\" (UniqueName: \"kubernetes.io/projected/fd79a6ce-4f90-49ed-8808-64a3f8e82398-kube-api-access-nbw4s\") pod \"authorino-operator-75cc8d5cb7-6jxtx\" (UID: \"fd79a6ce-4f90-49ed-8808-64a3f8e82398\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:36.740065 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.740033 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbw4s\" (UniqueName: \"kubernetes.io/projected/fd79a6ce-4f90-49ed-8808-64a3f8e82398-kube-api-access-nbw4s\") pod \"authorino-operator-75cc8d5cb7-6jxtx\" (UID: \"fd79a6ce-4f90-49ed-8808-64a3f8e82398\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:36.820294 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.820216 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:36.976496 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:36.976467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx"] Jul 06 14:21:36.980283 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:36.980256 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd79a6ce_4f90_49ed_8808_64a3f8e82398.slice/crio-ede28bb9bc3023ebcd705ea3f84290506df2d8d353cf41da6b8cd8a8ecea50b1 WatchSource:0}: Error finding container ede28bb9bc3023ebcd705ea3f84290506df2d8d353cf41da6b8cd8a8ecea50b1: Status 404 returned error can't find the container with id ede28bb9bc3023ebcd705ea3f84290506df2d8d353cf41da6b8cd8a8ecea50b1 Jul 06 14:21:37.572277 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:37.572236 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" event={"ID":"fd79a6ce-4f90-49ed-8808-64a3f8e82398","Type":"ContainerStarted","Data":"ede28bb9bc3023ebcd705ea3f84290506df2d8d353cf41da6b8cd8a8ecea50b1"} Jul 06 14:21:39.584583 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.584552 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" event={"ID":"fd79a6ce-4f90-49ed-8808-64a3f8e82398","Type":"ContainerStarted","Data":"d502ac38774a2feff7823bf9d242dd7db310bb8640246e04014a4374827fbaf2"} Jul 06 14:21:39.584955 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.584618 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:39.639783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.639727 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" podStartSLOduration=1.45309044 podStartE2EDuration="3.639708124s" totalImagesPullingTime="2.186617684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:39.636498001 +0000 UTC m=+415.331338249" watchObservedRunningTime="2026-07-06 14:21:39.639708124 +0000 UTC m=+415.334548372" Jul 06 14:21:39.656927 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.656894 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2"] Jul 06 14:21:39.660293 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.660277 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:39.663713 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:21:39.663684 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-operator-controller-manager-864dd78c4c-qw5t2\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="a3f47229-0c45-4948-9b09-3e1e48272f56" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:39.663878 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:21:39.663852 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-operator-controller-env\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" type="*v1.ConfigMap" Jul 06 14:21:39.665498 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:21:39.665472 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-5m8fn\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5m8fn\"" type="*v1.Secret" Jul 06 14:21:39.691521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.691487 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2"] Jul 06 14:21:39.747631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.747602 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwcll\" (UniqueName: \"kubernetes.io/projected/a3f47229-0c45-4948-9b09-3e1e48272f56-kube-api-access-qwcll\") pod \"dns-operator-controller-manager-864dd78c4c-qw5t2\" (UID: \"a3f47229-0c45-4948-9b09-3e1e48272f56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:39.848916 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.848837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwcll\" (UniqueName: \"kubernetes.io/projected/a3f47229-0c45-4948-9b09-3e1e48272f56-kube-api-access-qwcll\") pod \"dns-operator-controller-manager-864dd78c4c-qw5t2\" (UID: \"a3f47229-0c45-4948-9b09-3e1e48272f56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:39.865626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:39.865600 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwcll\" (UniqueName: \"kubernetes.io/projected/a3f47229-0c45-4948-9b09-3e1e48272f56-kube-api-access-qwcll\") pod \"dns-operator-controller-manager-864dd78c4c-qw5t2\" (UID: \"a3f47229-0c45-4948-9b09-3e1e48272f56\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:40.829235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:40.829189 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5m8fn\"" Jul 06 14:21:40.830254 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:40.830238 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:40.977496 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:40.977469 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2"] Jul 06 14:21:40.980637 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:40.980605 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f47229_0c45_4948_9b09_3e1e48272f56.slice/crio-e619b54c3d736e747b62c924fbe8447a376f2753b01ef84238bfa99ebd947b15 WatchSource:0}: Error finding container e619b54c3d736e747b62c924fbe8447a376f2753b01ef84238bfa99ebd947b15: Status 404 returned error can't find the container with id e619b54c3d736e747b62c924fbe8447a376f2753b01ef84238bfa99ebd947b15 Jul 06 14:21:41.260830 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:41.260802 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 14:21:41.596234 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:41.596141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" event={"ID":"a3f47229-0c45-4948-9b09-3e1e48272f56","Type":"ContainerStarted","Data":"e619b54c3d736e747b62c924fbe8447a376f2753b01ef84238bfa99ebd947b15"} Jul 06 14:21:42.446631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.446589 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8"] Jul 06 14:21:42.451025 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.450998 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:42.455758 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.455739 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-zhh4l\"" Jul 06 14:21:42.485229 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.485186 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8"] Jul 06 14:21:42.576010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.575978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btww5\" (UniqueName: \"kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5\") pod \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" (UID: \"bd4884d6-6796-43d5-ae8f-c4eb367d79b6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:42.677069 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.677029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btww5\" (UniqueName: \"kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5\") pod \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" (UID: \"bd4884d6-6796-43d5-ae8f-c4eb367d79b6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:42.709657 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.709630 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btww5\" (UniqueName: \"kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5\") pod \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" (UID: \"bd4884d6-6796-43d5-ae8f-c4eb367d79b6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:42.761369 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:42.761340 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:43.218164 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:43.218133 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4884d6_6796_43d5_ae8f_c4eb367d79b6.slice/crio-57f8d0786921d69069256f4189db2c1d269eb0fc2c130f45b2ddcec87aced3ba WatchSource:0}: Error finding container 57f8d0786921d69069256f4189db2c1d269eb0fc2c130f45b2ddcec87aced3ba: Status 404 returned error can't find the container with id 57f8d0786921d69069256f4189db2c1d269eb0fc2c130f45b2ddcec87aced3ba Jul 06 14:21:43.220122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:43.220104 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8"] Jul 06 14:21:43.607698 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:43.607665 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" event={"ID":"a3f47229-0c45-4948-9b09-3e1e48272f56","Type":"ContainerStarted","Data":"bc63371aff4f7523f72d9b748f3252498e5ca398a9613a98926419a649aaf6a3"} Jul 06 14:21:43.608124 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:43.607726 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:43.608979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:43.608952 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" event={"ID":"bd4884d6-6796-43d5-ae8f-c4eb367d79b6","Type":"ContainerStarted","Data":"57f8d0786921d69069256f4189db2c1d269eb0fc2c130f45b2ddcec87aced3ba"} Jul 06 14:21:43.634385 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:43.634336 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" podStartSLOduration=2.5289943900000003 podStartE2EDuration="4.634320858s" totalImagesPullingTime="2.105326468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:43.634106457 +0000 UTC m=+419.328946705" watchObservedRunningTime="2026-07-06 14:21:43.634320858 +0000 UTC m=+419.329161105" Jul 06 14:21:45.621728 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:45.621691 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" event={"ID":"bd4884d6-6796-43d5-ae8f-c4eb367d79b6","Type":"ContainerStarted","Data":"a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4"} Jul 06 14:21:45.622128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:45.621766 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:21:45.653509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:45.653463 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" podStartSLOduration=1.757747628 podStartE2EDuration="3.653449429s" totalImagesPullingTime="1.895701801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:45.651550339 +0000 UTC m=+421.346390586" watchObservedRunningTime="2026-07-06 14:21:45.653449429 +0000 UTC m=+421.348289676" Jul 06 14:21:46.386221 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.386173 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc"] Jul 06 14:21:46.389451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.389436 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.396640 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.396618 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6vfj7\"" Jul 06 14:21:46.438785 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.438751 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc"] Jul 06 14:21:46.508691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.508661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phxxz\" (UniqueName: \"kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.508851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.508725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.609426 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.609393 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.609599 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.609445 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phxxz\" (UniqueName: \"kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.609858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.609836 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.643413 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.643337 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phxxz\" (UniqueName: \"kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.699088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.699059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:46.854616 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:46.854592 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc"] Jul 06 14:21:46.856566 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:21:46.856542 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33568c2f_b47c_4ccd_8a3c_4ee601b37d23.slice/crio-92439094e2b02624552cfb5907176992df7fbe59208bfcf455bbc51f0bc3c86a WatchSource:0}: Error finding container 92439094e2b02624552cfb5907176992df7fbe59208bfcf455bbc51f0bc3c86a: Status 404 returned error can't find the container with id 92439094e2b02624552cfb5907176992df7fbe59208bfcf455bbc51f0bc3c86a Jul 06 14:21:47.633594 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:47.633559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" event={"ID":"33568c2f-b47c-4ccd-8a3c-4ee601b37d23","Type":"ContainerStarted","Data":"92439094e2b02624552cfb5907176992df7fbe59208bfcf455bbc51f0bc3c86a"} Jul 06 14:21:50.591805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:50.591771 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-6jxtx" Jul 06 14:21:53.664596 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:53.664495 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" event={"ID":"33568c2f-b47c-4ccd-8a3c-4ee601b37d23","Type":"ContainerStarted","Data":"d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666"} Jul 06 14:21:53.664596 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:53.664565 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:21:53.700657 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:53.700604 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" podStartSLOduration=1.171340788 podStartE2EDuration="7.700588497s" totalImagesPullingTime="6.529247709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:21:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:21:53.698083882 +0000 UTC m=+429.392924140" watchObservedRunningTime="2026-07-06 14:21:53.700588497 +0000 UTC m=+429.395428744" Jul 06 14:21:54.617606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:54.617579 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qw5t2" Jul 06 14:21:56.629688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:21:56.629657 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:04.671359 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:04.671326 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:06.861615 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.861530 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc"] Jul 06 14:22:06.862000 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.861720 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" containerName="manager" containerID="cri-o://d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666" gracePeriod=2 Jul 06 14:22:06.871810 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.871782 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc"] Jul 06 14:22:06.906202 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.906167 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8"] Jul 06 14:22:06.906486 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.906433 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" containerName="manager" containerID="cri-o://a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4" gracePeriod=2 Jul 06 14:22:06.911004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:06.910978 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8"] Jul 06 14:22:06.913799 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:06.913767 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:06.918589 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:06.918540 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.017035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.016994 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:07.017519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017496 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" containerName="manager" Jul 06 14:22:07.017627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017522 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" containerName="manager" Jul 06 14:22:07.017627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017550 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" containerName="manager" Jul 06 14:22:07.017627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017558 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" containerName="manager" Jul 06 14:22:07.017775 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017648 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" containerName="manager" Jul 06 14:22:07.017775 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.017664 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" containerName="manager" Jul 06 14:22:07.020864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.020845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.026699 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.026671 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv"] Jul 06 14:22:07.030433 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.030410 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.032506 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.032472 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.037138 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.037105 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.064081 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.063950 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv"] Jul 06 14:22:07.077619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.077592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.077743 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.077640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd6cz\" (UniqueName: \"kubernetes.io/projected/d5ea7f07-7970-48e7-8662-08385f6ce910-kube-api-access-nd6cz\") pod \"limitador-operator-controller-manager-859d7b78bd-nfblv\" (UID: \"d5ea7f07-7970-48e7-8662-08385f6ce910\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.077743 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.077690 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lndfq\" (UniqueName: \"kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.084766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.084742 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:07.147395 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.147355 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.152229 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.152169 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.178079 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.178055 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.178237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.178084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.178237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.178139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd6cz\" (UniqueName: \"kubernetes.io/projected/d5ea7f07-7970-48e7-8662-08385f6ce910-kube-api-access-nd6cz\") pod \"limitador-operator-controller-manager-859d7b78bd-nfblv\" (UID: \"d5ea7f07-7970-48e7-8662-08385f6ce910\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.178237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.178167 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lndfq\" (UniqueName: \"kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.178464 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.178438 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.180939 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.180919 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.196526 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.196500 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.199802 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.199779 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.202835 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.202812 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.203971 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.203955 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lndfq\" (UniqueName: \"kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-99ldn\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.205848 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.205824 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.207215 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.207182 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd6cz\" (UniqueName: \"kubernetes.io/projected/d5ea7f07-7970-48e7-8662-08385f6ce910-kube-api-access-nd6cz\") pod \"limitador-operator-controller-manager-859d7b78bd-nfblv\" (UID: \"d5ea7f07-7970-48e7-8662-08385f6ce910\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.279364 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.279340 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btww5\" (UniqueName: \"kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5\") pod \"bd4884d6-6796-43d5-ae8f-c4eb367d79b6\" (UID: \"bd4884d6-6796-43d5-ae8f-c4eb367d79b6\") " Jul 06 14:22:07.279575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.279406 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phxxz\" (UniqueName: \"kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz\") pod \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " Jul 06 14:22:07.279575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.279459 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume\") pod \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\" (UID: \"33568c2f-b47c-4ccd-8a3c-4ee601b37d23\") " Jul 06 14:22:07.279947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.279926 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "33568c2f-b47c-4ccd-8a3c-4ee601b37d23" (UID: "33568c2f-b47c-4ccd-8a3c-4ee601b37d23"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:22:07.281354 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.281336 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5" (OuterVolumeSpecName: "kube-api-access-btww5") pod "bd4884d6-6796-43d5-ae8f-c4eb367d79b6" (UID: "bd4884d6-6796-43d5-ae8f-c4eb367d79b6"). InnerVolumeSpecName "kube-api-access-btww5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:22:07.281458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.281440 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz" (OuterVolumeSpecName: "kube-api-access-phxxz") pod "33568c2f-b47c-4ccd-8a3c-4ee601b37d23" (UID: "33568c2f-b47c-4ccd-8a3c-4ee601b37d23"). InnerVolumeSpecName "kube-api-access-phxxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:22:07.336488 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.336460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.345591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.345571 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.380677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.380595 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-extensions-socket-volume\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:07.380677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.380627 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btww5\" (UniqueName: \"kubernetes.io/projected/bd4884d6-6796-43d5-ae8f-c4eb367d79b6-kube-api-access-btww5\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:07.380677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.380636 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phxxz\" (UniqueName: \"kubernetes.io/projected/33568c2f-b47c-4ccd-8a3c-4ee601b37d23-kube-api-access-phxxz\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:07.537426 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.537374 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv"] Jul 06 14:22:07.539712 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.539684 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:07.540156 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:07.540133 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ea7f07_7970_48e7_8662_08385f6ce910.slice/crio-20639cbf91936b0786e4657c4426012e6d507c0ee36bd67b4bcf79ec1c680238 WatchSource:0}: Error finding container 20639cbf91936b0786e4657c4426012e6d507c0ee36bd67b4bcf79ec1c680238: Status 404 returned error can't find the container with id 20639cbf91936b0786e4657c4426012e6d507c0ee36bd67b4bcf79ec1c680238 Jul 06 14:22:07.540813 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:07.540746 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d5b32f_de1c_42b3_b648_f1f44d6e05b8.slice/crio-eca533923ff6e58c22a0d9573a1c82e3da46479d5214839e49795f04599d86d5 WatchSource:0}: Error finding container eca533923ff6e58c22a0d9573a1c82e3da46479d5214839e49795f04599d86d5: Status 404 returned error can't find the container with id eca533923ff6e58c22a0d9573a1c82e3da46479d5214839e49795f04599d86d5 Jul 06 14:22:07.736782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.736726 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" event={"ID":"d5ea7f07-7970-48e7-8662-08385f6ce910","Type":"ContainerStarted","Data":"fa3d820c1b7c4049cd5fed02345e627d4e1c56267ba1a728b371a80f89796cdb"} Jul 06 14:22:07.737010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.736805 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:07.737010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.736829 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" event={"ID":"d5ea7f07-7970-48e7-8662-08385f6ce910","Type":"ContainerStarted","Data":"20639cbf91936b0786e4657c4426012e6d507c0ee36bd67b4bcf79ec1c680238"} Jul 06 14:22:07.738408 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.738378 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" event={"ID":"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8","Type":"ContainerStarted","Data":"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c"} Jul 06 14:22:07.738562 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.738432 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" event={"ID":"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8","Type":"ContainerStarted","Data":"eca533923ff6e58c22a0d9573a1c82e3da46479d5214839e49795f04599d86d5"} Jul 06 14:22:07.738562 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.738457 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:07.739654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.739630 2581 generic.go:356] "Generic (PLEG): container finished" podID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" containerID="a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4" exitCode=0 Jul 06 14:22:07.739759 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.739659 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.739759 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.739720 2581 scope.go:117] "RemoveContainer" containerID="a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4" Jul 06 14:22:07.740976 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.740954 2581 generic.go:356] "Generic (PLEG): container finished" podID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" containerID="d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666" exitCode=0 Jul 06 14:22:07.741084 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.741047 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.742700 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.742678 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.750285 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.748506 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:07.752124 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.752103 2581 scope.go:117] "RemoveContainer" containerID="a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4" Jul 06 14:22:07.752767 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.752738 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4\": container with ID starting with a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4 not found: ID does not exist" containerID="a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4" Jul 06 14:22:07.752850 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.752774 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4"} err="failed to get container status \"a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4\": rpc error: code = NotFound desc = could not find container \"a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4\": container with ID starting with a8a372911cf115442fd1d0853e6d94fcb5ed6630d692c3ad3b2e5ce24afce1a4 not found: ID does not exist" Jul 06 14:22:07.752850 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.752795 2581 scope.go:117] "RemoveContainer" containerID="d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666" Jul 06 14:22:07.761266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.761248 2581 scope.go:117] "RemoveContainer" containerID="d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666" Jul 06 14:22:07.761510 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.761493 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666\": container with ID starting with d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666 not found: ID does not exist" containerID="d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666" Jul 06 14:22:07.761556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.761518 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666"} err="failed to get container status \"d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666\": rpc error: code = NotFound desc = could not find container \"d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666\": container with ID starting with d1de29c3b69ed7002bc96a096cfb1aacaea2126d98f6a67bf64cf91246bbe666 not found: ID does not exist" Jul 06 14:22:07.902654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.902608 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" podStartSLOduration=1.902589607 podStartE2EDuration="1.902589607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:07.818793639 +0000 UTC m=+443.513633884" watchObservedRunningTime="2026-07-06 14:22:07.902589607 +0000 UTC m=+443.597429844" Jul 06 14:22:07.903057 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:07.903001 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" podStartSLOduration=1.902991354 podStartE2EDuration="1.902991354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:07.902538546 +0000 UTC m=+443.597378794" watchObservedRunningTime="2026-07-06 14:22:07.902991354 +0000 UTC m=+443.597831601" Jul 06 14:22:07.925704 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.925670 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-nwsj8\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nwsj8" Jul 06 14:22:07.935246 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:07.935217 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-jznvc\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jznvc" Jul 06 14:22:08.813814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:08.813783 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33568c2f-b47c-4ccd-8a3c-4ee601b37d23" path="/var/lib/kubelet/pods/33568c2f-b47c-4ccd-8a3c-4ee601b37d23/volumes" Jul 06 14:22:08.814098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:08.814084 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4884d6-6796-43d5-ae8f-c4eb367d79b6" path="/var/lib/kubelet/pods/bd4884d6-6796-43d5-ae8f-c4eb367d79b6/volumes" Jul 06 14:22:11.306507 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.306474 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:22:11.313957 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.313938 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.356246 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.356222 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:22:11.415840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.415814 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gplcm\" (UniqueName: \"kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.415981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.415877 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.517110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.517071 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.517394 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.517122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gplcm\" (UniqueName: \"kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.517551 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.517527 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.536271 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.536236 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gplcm\" (UniqueName: \"kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm\") pod \"kuadrant-operator-controller-manager-86878c769f-jh5wn\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.623488 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.623410 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:11.797326 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:11.797301 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:22:11.798065 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:11.798041 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d57a80b_9fb5_4c4d_8f2e_ab27e05653f1.slice/crio-6703b4d536339e8f0af908d280d21b948f31ab395dfc0cd0e9663ca92fc3f3e3 WatchSource:0}: Error finding container 6703b4d536339e8f0af908d280d21b948f31ab395dfc0cd0e9663ca92fc3f3e3: Status 404 returned error can't find the container with id 6703b4d536339e8f0af908d280d21b948f31ab395dfc0cd0e9663ca92fc3f3e3 Jul 06 14:22:12.769450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:12.769410 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" event={"ID":"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1","Type":"ContainerStarted","Data":"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6"} Jul 06 14:22:12.769884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:12.769470 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" event={"ID":"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1","Type":"ContainerStarted","Data":"6703b4d536339e8f0af908d280d21b948f31ab395dfc0cd0e9663ca92fc3f3e3"} Jul 06 14:22:12.769884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:12.769494 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:18.749925 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:18.749898 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-nfblv" Jul 06 14:22:18.750298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:18.749971 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:18.798258 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:18.798177 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" podStartSLOduration=7.798157263 podStartE2EDuration="7.798157263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:12.806344468 +0000 UTC m=+448.501184715" watchObservedRunningTime="2026-07-06 14:22:18.798157263 +0000 UTC m=+454.492997508" Jul 06 14:22:23.777762 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:23.777734 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:22:23.882078 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:23.882043 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:23.882285 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:23.882264 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" podUID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" containerName="manager" containerID="cri-o://e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c" gracePeriod=10 Jul 06 14:22:24.124385 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.124360 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:24.221803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.221772 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lndfq\" (UniqueName: \"kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq\") pod \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " Jul 06 14:22:24.221986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.221841 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume\") pod \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\" (UID: \"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8\") " Jul 06 14:22:24.222281 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.222250 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" (UID: "f9d5b32f-de1c-42b3-b648-f1f44d6e05b8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:22:24.223909 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.223890 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq" (OuterVolumeSpecName: "kube-api-access-lndfq") pod "f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" (UID: "f9d5b32f-de1c-42b3-b648-f1f44d6e05b8"). InnerVolumeSpecName "kube-api-access-lndfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:22:24.322642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.322577 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lndfq\" (UniqueName: \"kubernetes.io/projected/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-kube-api-access-lndfq\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:24.322642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.322602 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8-extensions-socket-volume\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:24.840639 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.840609 2581 generic.go:356] "Generic (PLEG): container finished" podID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" containerID="e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c" exitCode=0 Jul 06 14:22:24.841013 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.840653 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" event={"ID":"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8","Type":"ContainerDied","Data":"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c"} Jul 06 14:22:24.841013 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.840658 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" Jul 06 14:22:24.841013 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.840711 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn" event={"ID":"f9d5b32f-de1c-42b3-b648-f1f44d6e05b8","Type":"ContainerDied","Data":"eca533923ff6e58c22a0d9573a1c82e3da46479d5214839e49795f04599d86d5"} Jul 06 14:22:24.841013 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.840731 2581 scope.go:117] "RemoveContainer" containerID="e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c" Jul 06 14:22:24.849071 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.849055 2581 scope.go:117] "RemoveContainer" containerID="e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c" Jul 06 14:22:24.849334 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:24.849317 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c\": container with ID starting with e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c not found: ID does not exist" containerID="e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c" Jul 06 14:22:24.849394 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.849342 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c"} err="failed to get container status \"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c\": rpc error: code = NotFound desc = could not find container \"e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c\": container with ID starting with e911552580a65cfe6fef4b31c15b61eb32e9b5f5015db22d8288d2d697476b6c not found: ID does not exist" Jul 06 14:22:24.877150 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.877110 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:24.890011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:24.889983 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-99ldn"] Jul 06 14:22:26.813716 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:26.813682 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" path="/var/lib/kubelet/pods/f9d5b32f-de1c-42b3-b648-f1f44d6e05b8/volumes" Jul 06 14:22:40.767041 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.767004 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62"] Jul 06 14:22:40.767523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.767364 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" containerName="manager" Jul 06 14:22:40.767523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.767375 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" containerName="manager" Jul 06 14:22:40.767523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.767429 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d5b32f-de1c-42b3-b648-f1f44d6e05b8" containerName="manager" Jul 06 14:22:40.769462 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.769447 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.773148 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:40.773119 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-default-gateway-openshift-default-56654dcf7-psc62\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" podUID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.773314 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:40.773152 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-default-gateway-openshift-default-dockercfg-9j7hq\" is forbidden: User \"system:node:ip-10-0-131-65.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-65.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9j7hq\"" type="*v1.Secret" Jul 06 14:22:40.804176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.804152 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62"] Jul 06 14:22:40.867836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.867806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4fm\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-kube-api-access-sr4fm\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.867836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.867841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.867915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.867931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.867986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.868011 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868143 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.868053 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868143 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.868071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.868143 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.868093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969584 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.969787 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4fm\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-kube-api-access-sr4fm\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.969869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970130 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.970105 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970320 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.970295 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.970533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970662 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.970575 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.970724 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.970705 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.971949 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.971932 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.972265 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.972247 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:40.989999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:40.989972 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4fm\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-kube-api-access-sr4fm\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:41.004544 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:41.004524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37bf8f08-e9ec-4541-ad03-b20618f0a6fd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-psc62\" (UID: \"37bf8f08-e9ec-4541-ad03-b20618f0a6fd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:42.030848 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.030814 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9j7hq\"" Jul 06 14:22:42.040163 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.040131 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:42.183298 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.183263 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62"] Jul 06 14:22:42.188127 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:42.188093 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37bf8f08_e9ec_4541_ad03_b20618f0a6fd.slice/crio-bf5d623857d8ae0e935a56c2190a6f9789ac3dd428fd3edb2c4e9f13b46bf634 WatchSource:0}: Error finding container bf5d623857d8ae0e935a56c2190a6f9789ac3dd428fd3edb2c4e9f13b46bf634: Status 404 returned error can't find the container with id bf5d623857d8ae0e935a56c2190a6f9789ac3dd428fd3edb2c4e9f13b46bf634 Jul 06 14:22:42.190615 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.190576 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:22:42.190718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.190651 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:22:42.190718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.190687 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:22:42.934691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.934660 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" event={"ID":"37bf8f08-e9ec-4541-ad03-b20618f0a6fd","Type":"ContainerStarted","Data":"b13ae3d934880a8b19cc64b5a81fa38d8ecf7bdc744f515115f0cbe4ed519494"} Jul 06 14:22:42.934871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.934709 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" event={"ID":"37bf8f08-e9ec-4541-ad03-b20618f0a6fd","Type":"ContainerStarted","Data":"bf5d623857d8ae0e935a56c2190a6f9789ac3dd428fd3edb2c4e9f13b46bf634"} Jul 06 14:22:42.978334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:42.978278 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" podStartSLOduration=2.978261637 podStartE2EDuration="2.978261637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:42.977656929 +0000 UTC m=+478.672497201" watchObservedRunningTime="2026-07-06 14:22:42.978261637 +0000 UTC m=+478.673101896" Jul 06 14:22:43.040453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:43.040421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:43.045136 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:43.045116 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:43.940176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:43.940146 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:43.941310 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:43.941285 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:22:44.990623 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:44.990594 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:44.994033 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:44.994015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:44.997297 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:44.997280 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9b6w6\"" Jul 06 14:22:45.009005 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.008984 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:45.107997 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.107961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8lxr\" (UniqueName: \"kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr\") pod \"authorino-58cd955675-5qgnd\" (UID: \"511046c9-524d-4019-b9a7-0035fc2c5374\") " pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:45.208897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.208861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8lxr\" (UniqueName: \"kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr\") pod \"authorino-58cd955675-5qgnd\" (UID: \"511046c9-524d-4019-b9a7-0035fc2c5374\") " pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:45.228459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.228435 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8lxr\" (UniqueName: \"kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr\") pod \"authorino-58cd955675-5qgnd\" (UID: \"511046c9-524d-4019-b9a7-0035fc2c5374\") " pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:45.238852 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.238827 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:22:45.241340 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.241289 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:22:45.253452 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.253431 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:22:45.302690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.302663 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:45.410874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.410830 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9q78\" (UniqueName: \"kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78\") pod \"authorino-75d556bd77-x496m\" (UID: \"350ee84c-9a14-4250-b917-3854e99e35ad\") " pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:22:45.444221 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.444183 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:45.446563 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:45.446534 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod511046c9_524d_4019_b9a7_0035fc2c5374.slice/crio-ef5adf00522c4a35e1400f6d511aedd5acdb5aaa813f3973b838a92d0255958a WatchSource:0}: Error finding container ef5adf00522c4a35e1400f6d511aedd5acdb5aaa813f3973b838a92d0255958a: Status 404 returned error can't find the container with id ef5adf00522c4a35e1400f6d511aedd5acdb5aaa813f3973b838a92d0255958a Jul 06 14:22:45.511626 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.511535 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9q78\" (UniqueName: \"kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78\") pod \"authorino-75d556bd77-x496m\" (UID: \"350ee84c-9a14-4250-b917-3854e99e35ad\") " pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:22:45.520819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.520783 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9q78\" (UniqueName: \"kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78\") pod \"authorino-75d556bd77-x496m\" (UID: \"350ee84c-9a14-4250-b917-3854e99e35ad\") " pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:22:45.550412 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.550384 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:22:45.677367 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.677341 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:22:45.679186 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:22:45.679162 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod350ee84c_9a14_4250_b917_3854e99e35ad.slice/crio-ef8abc920c5d7acb30aabac656820d039301170b3865aad91eece897632ee3ef WatchSource:0}: Error finding container ef8abc920c5d7acb30aabac656820d039301170b3865aad91eece897632ee3ef: Status 404 returned error can't find the container with id ef8abc920c5d7acb30aabac656820d039301170b3865aad91eece897632ee3ef Jul 06 14:22:45.950991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.950955 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x496m" event={"ID":"350ee84c-9a14-4250-b917-3854e99e35ad","Type":"ContainerStarted","Data":"ef8abc920c5d7acb30aabac656820d039301170b3865aad91eece897632ee3ef"} Jul 06 14:22:45.952007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:45.951976 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5qgnd" event={"ID":"511046c9-524d-4019-b9a7-0035fc2c5374","Type":"ContainerStarted","Data":"ef5adf00522c4a35e1400f6d511aedd5acdb5aaa813f3973b838a92d0255958a"} Jul 06 14:22:49.982111 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:49.982076 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x496m" event={"ID":"350ee84c-9a14-4250-b917-3854e99e35ad","Type":"ContainerStarted","Data":"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9"} Jul 06 14:22:49.983595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:49.983574 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5qgnd" event={"ID":"511046c9-524d-4019-b9a7-0035fc2c5374","Type":"ContainerStarted","Data":"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281"} Jul 06 14:22:50.005594 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:50.005548 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-x496m" podStartSLOduration=1.434066334 podStartE2EDuration="5.005535218s" totalImagesPullingTime="3.571468884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:50.003123135 +0000 UTC m=+485.697963392" watchObservedRunningTime="2026-07-06 14:22:50.005535218 +0000 UTC m=+485.700375486" Jul 06 14:22:50.032257 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:50.032181 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5qgnd" podStartSLOduration=2.239428633 podStartE2EDuration="6.03216647s" totalImagesPullingTime="3.792737837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:22:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:22:50.031560236 +0000 UTC m=+485.726400482" watchObservedRunningTime="2026-07-06 14:22:50.03216647 +0000 UTC m=+485.727006722" Jul 06 14:22:50.081838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:50.081804 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:51.993235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:51.993181 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5qgnd" podUID="511046c9-524d-4019-b9a7-0035fc2c5374" containerName="authorino" containerID="cri-o://7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281" gracePeriod=30 Jul 06 14:22:52.241004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.240984 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:52.378824 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.378750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8lxr\" (UniqueName: \"kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr\") pod \"511046c9-524d-4019-b9a7-0035fc2c5374\" (UID: \"511046c9-524d-4019-b9a7-0035fc2c5374\") " Jul 06 14:22:52.380864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.380841 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr" (OuterVolumeSpecName: "kube-api-access-f8lxr") pod "511046c9-524d-4019-b9a7-0035fc2c5374" (UID: "511046c9-524d-4019-b9a7-0035fc2c5374"). InnerVolumeSpecName "kube-api-access-f8lxr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:22:52.479395 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.479360 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8lxr\" (UniqueName: \"kubernetes.io/projected/511046c9-524d-4019-b9a7-0035fc2c5374-kube-api-access-f8lxr\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:22:52.998572 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.998541 2581 generic.go:356] "Generic (PLEG): container finished" podID="511046c9-524d-4019-b9a7-0035fc2c5374" containerID="7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281" exitCode=0 Jul 06 14:22:52.998991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.998582 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5qgnd" Jul 06 14:22:52.998991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.998632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5qgnd" event={"ID":"511046c9-524d-4019-b9a7-0035fc2c5374","Type":"ContainerDied","Data":"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281"} Jul 06 14:22:52.998991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.998680 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5qgnd" event={"ID":"511046c9-524d-4019-b9a7-0035fc2c5374","Type":"ContainerDied","Data":"ef5adf00522c4a35e1400f6d511aedd5acdb5aaa813f3973b838a92d0255958a"} Jul 06 14:22:52.998991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:52.998705 2581 scope.go:117] "RemoveContainer" containerID="7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281" Jul 06 14:22:53.006876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:53.006859 2581 scope.go:117] "RemoveContainer" containerID="7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281" Jul 06 14:22:53.007147 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:22:53.007112 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281\": container with ID starting with 7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281 not found: ID does not exist" containerID="7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281" Jul 06 14:22:53.007217 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:53.007156 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281"} err="failed to get container status \"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281\": rpc error: code = NotFound desc = could not find container \"7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281\": container with ID starting with 7eff4c8d2b9f2d9f15dc2d1c07f0d2937c474225f3ac2d427f7d13b6f1f06281 not found: ID does not exist" Jul 06 14:22:53.018940 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:53.018917 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:53.022318 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:53.022300 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5qgnd"] Jul 06 14:22:54.814832 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:22:54.814802 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511046c9-524d-4019-b9a7-0035fc2c5374" path="/var/lib/kubelet/pods/511046c9-524d-4019-b9a7-0035fc2c5374/volumes" Jul 06 14:23:26.402105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.402064 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4"] Jul 06 14:23:26.402679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.402641 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511046c9-524d-4019-b9a7-0035fc2c5374" containerName="authorino" Jul 06 14:23:26.402679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.402661 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="511046c9-524d-4019-b9a7-0035fc2c5374" containerName="authorino" Jul 06 14:23:26.402792 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.402772 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="511046c9-524d-4019-b9a7-0035fc2c5374" containerName="authorino" Jul 06 14:23:26.407132 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.407109 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.412502 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.412480 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 14:23:26.412673 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.412654 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5g2bj\"" Jul 06 14:23:26.413552 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.413535 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 14:23:26.421433 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.421409 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4"] Jul 06 14:23:26.476619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.476586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.476793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.476623 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.476793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.476712 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7vp6\" (UniqueName: \"kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.577591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.577555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7vp6\" (UniqueName: \"kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.577765 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.577634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.577765 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.577662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.578049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.578028 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.578049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.578041 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.590551 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.590521 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7vp6\" (UniqueName: \"kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6\") pod \"2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.715942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.715915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:26.866855 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:26.866831 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4"] Jul 06 14:23:26.869050 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:23:26.869022 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ea56cdf_2e88_4b03_8b33_4ca6e593601f.slice/crio-486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312 WatchSource:0}: Error finding container 486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312: Status 404 returned error can't find the container with id 486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312 Jul 06 14:23:27.175564 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:27.175527 2581 generic.go:356] "Generic (PLEG): container finished" podID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerID="d3a7543742d3d65c8d9b950bfc8f852b1e63c930980e38f2642091a98861a166" exitCode=0 Jul 06 14:23:27.175728 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:27.175617 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" event={"ID":"2ea56cdf-2e88-4b03-8b33-4ca6e593601f","Type":"ContainerDied","Data":"d3a7543742d3d65c8d9b950bfc8f852b1e63c930980e38f2642091a98861a166"} Jul 06 14:23:27.175728 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:27.175671 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" event={"ID":"2ea56cdf-2e88-4b03-8b33-4ca6e593601f","Type":"ContainerStarted","Data":"486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312"} Jul 06 14:23:28.182405 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:28.182321 2581 generic.go:356] "Generic (PLEG): container finished" podID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerID="8f74f816dd8151dd69c7f5b8c1d47921a91229bc11608c9246cb9d0e962a1bd5" exitCode=0 Jul 06 14:23:28.182762 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:28.182416 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" event={"ID":"2ea56cdf-2e88-4b03-8b33-4ca6e593601f","Type":"ContainerDied","Data":"8f74f816dd8151dd69c7f5b8c1d47921a91229bc11608c9246cb9d0e962a1bd5"} Jul 06 14:23:29.189643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:29.189606 2581 generic.go:356] "Generic (PLEG): container finished" podID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerID="8ee3dec5726a04520698a82ea33ea377223cba13149d84eef7b109dae79fd245" exitCode=0 Jul 06 14:23:29.190077 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:29.189694 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" event={"ID":"2ea56cdf-2e88-4b03-8b33-4ca6e593601f","Type":"ContainerDied","Data":"8ee3dec5726a04520698a82ea33ea377223cba13149d84eef7b109dae79fd245"} Jul 06 14:23:30.327067 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.327043 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:30.409163 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.409136 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util\") pod \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " Jul 06 14:23:30.409163 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.409171 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle\") pod \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " Jul 06 14:23:30.409380 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.409259 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7vp6\" (UniqueName: \"kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6\") pod \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\" (UID: \"2ea56cdf-2e88-4b03-8b33-4ca6e593601f\") " Jul 06 14:23:30.409682 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.409655 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle" (OuterVolumeSpecName: "bundle") pod "2ea56cdf-2e88-4b03-8b33-4ca6e593601f" (UID: "2ea56cdf-2e88-4b03-8b33-4ca6e593601f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:23:30.411521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.411488 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6" (OuterVolumeSpecName: "kube-api-access-q7vp6") pod "2ea56cdf-2e88-4b03-8b33-4ca6e593601f" (UID: "2ea56cdf-2e88-4b03-8b33-4ca6e593601f"). InnerVolumeSpecName "kube-api-access-q7vp6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:23:30.414876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.414844 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util" (OuterVolumeSpecName: "util") pod "2ea56cdf-2e88-4b03-8b33-4ca6e593601f" (UID: "2ea56cdf-2e88-4b03-8b33-4ca6e593601f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:23:30.510157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.510053 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7vp6\" (UniqueName: \"kubernetes.io/projected/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-kube-api-access-q7vp6\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:23:30.510157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.510101 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-util\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:23:30.510157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:30.510114 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2ea56cdf-2e88-4b03-8b33-4ca6e593601f-bundle\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:23:31.208371 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:31.208336 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" event={"ID":"2ea56cdf-2e88-4b03-8b33-4ca6e593601f","Type":"ContainerDied","Data":"486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312"} Jul 06 14:23:31.208573 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:31.208386 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="486f75b18fd2c299715530190552d53950c92df8eb2d3cf7ae398a76880a3312" Jul 06 14:23:31.208573 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:31.208343 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2736463c5a4e6691c7a6ec457423dfda2e5f78fc1801325ffed26f71c8wntw4" Jul 06 14:23:51.662393 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662365 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:23:51.662748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662724 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="util" Jul 06 14:23:51.662748 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662735 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="util" Jul 06 14:23:51.662831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662750 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="extract" Jul 06 14:23:51.662831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662755 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="extract" Jul 06 14:23:51.662831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662766 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="pull" Jul 06 14:23:51.662831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662772 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="pull" Jul 06 14:23:51.662831 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.662823 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ea56cdf-2e88-4b03-8b33-4ca6e593601f" containerName="extract" Jul 06 14:23:51.666860 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.666841 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:51.670339 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.670311 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 06 14:23:51.670586 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.670570 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-6j574\"" Jul 06 14:23:51.671813 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.671797 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 06 14:23:51.671867 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.671833 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 06 14:23:51.679786 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.679763 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:23:51.697171 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.697141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tz62\" (UniqueName: \"kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62\") pod \"maas-keycloak-0\" (UID: \"258e6da1-d347-48c7-8856-504e9152529b\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:51.798090 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.798052 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tz62\" (UniqueName: \"kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62\") pod \"maas-keycloak-0\" (UID: \"258e6da1-d347-48c7-8856-504e9152529b\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:51.809770 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.809741 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tz62\" (UniqueName: \"kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62\") pod \"maas-keycloak-0\" (UID: \"258e6da1-d347-48c7-8856-504e9152529b\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:51.976476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:51.976443 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:52.118561 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:52.118536 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:23:52.120730 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:23:52.120701 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod258e6da1_d347_48c7_8856_504e9152529b.slice/crio-3d92a5cd44ed1df4ed3b0e636f6f26cdb8bea9996ed59fe92655dfa921cc0f4c WatchSource:0}: Error finding container 3d92a5cd44ed1df4ed3b0e636f6f26cdb8bea9996ed59fe92655dfa921cc0f4c: Status 404 returned error can't find the container with id 3d92a5cd44ed1df4ed3b0e636f6f26cdb8bea9996ed59fe92655dfa921cc0f4c Jul 06 14:23:52.321506 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:52.321418 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"258e6da1-d347-48c7-8856-504e9152529b","Type":"ContainerStarted","Data":"3d92a5cd44ed1df4ed3b0e636f6f26cdb8bea9996ed59fe92655dfa921cc0f4c"} Jul 06 14:23:57.358210 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:57.358150 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"258e6da1-d347-48c7-8856-504e9152529b","Type":"ContainerStarted","Data":"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e"} Jul 06 14:23:57.383565 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:57.383507 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.5779920459999999 podStartE2EDuration="6.383490876s" totalImagesPullingTime="4.80549883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:23:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:23:57.383277478 +0000 UTC m=+553.078117726" watchObservedRunningTime="2026-07-06 14:23:57.383490876 +0000 UTC m=+553.078331124" Jul 06 14:23:57.976764 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:57.976726 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 06 14:23:57.978113 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:57.978067 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:23:58.977001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:58.976958 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:23:59.977652 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:23:59.977604 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:00.977990 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:00.977938 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:01.977688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:01.977636 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:01.978814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:01.978143 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 06 14:24:02.977677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:02.977631 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:03.977777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:03.977725 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:04.977453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:04.977391 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:05.977900 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:05.977845 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:06.977343 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:06.977283 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:07.977424 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:07.977384 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:08.977703 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:08.977657 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 06 14:24:10.072906 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:10.072874 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 06 14:24:10.088616 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:10.088562 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:24:20.078800 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:20.078768 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 06 14:24:21.046214 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.046163 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:21.053792 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.053771 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:21.060724 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.060705 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:21.187697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.187661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pwdm\" (UniqueName: \"kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm\") pod \"authorino-5d4b67c446-gs2sd\" (UID: \"b10f118c-6087-4d73-a2e9-33aa1d40d50b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:21.288405 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.288364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pwdm\" (UniqueName: \"kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm\") pod \"authorino-5d4b67c446-gs2sd\" (UID: \"b10f118c-6087-4d73-a2e9-33aa1d40d50b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:21.299271 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.299187 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pwdm\" (UniqueName: \"kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm\") pod \"authorino-5d4b67c446-gs2sd\" (UID: \"b10f118c-6087-4d73-a2e9-33aa1d40d50b\") " pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:21.300290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.300271 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:21.300487 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.300477 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:21.323320 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.323282 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-854d59684-h9zht"] Jul 06 14:24:21.326074 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.326051 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:21.336777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.336755 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-854d59684-h9zht"] Jul 06 14:24:21.448416 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.448387 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:21.450829 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:24:21.450805 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb10f118c_6087_4d73_a2e9_33aa1d40d50b.slice/crio-733a7a66aff80ff1f8b1a1607b2ae27dec1bb28f32bcab6f1420becb6a4129fa WatchSource:0}: Error finding container 733a7a66aff80ff1f8b1a1607b2ae27dec1bb28f32bcab6f1420becb6a4129fa: Status 404 returned error can't find the container with id 733a7a66aff80ff1f8b1a1607b2ae27dec1bb28f32bcab6f1420becb6a4129fa Jul 06 14:24:21.489939 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.489915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llxvc\" (UniqueName: \"kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc\") pod \"authorino-854d59684-h9zht\" (UID: \"0c0bf967-a0f8-43ee-a413-e0869b0433b2\") " pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:21.527459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.527424 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" event={"ID":"b10f118c-6087-4d73-a2e9-33aa1d40d50b","Type":"ContainerStarted","Data":"733a7a66aff80ff1f8b1a1607b2ae27dec1bb28f32bcab6f1420becb6a4129fa"} Jul 06 14:24:21.542904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.542871 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-854d59684-h9zht"] Jul 06 14:24:21.543149 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:21.543130 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-llxvc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-854d59684-h9zht" podUID="0c0bf967-a0f8-43ee-a413-e0869b0433b2" Jul 06 14:24:21.577179 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.577108 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:24:21.580240 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.580217 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.584381 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.584359 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 14:24:21.590719 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.590697 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:24:21.591097 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.591077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llxvc\" (UniqueName: \"kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc\") pod \"authorino-854d59684-h9zht\" (UID: \"0c0bf967-a0f8-43ee-a413-e0869b0433b2\") " pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:21.604631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.604600 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llxvc\" (UniqueName: \"kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc\") pod \"authorino-854d59684-h9zht\" (UID: \"0c0bf967-a0f8-43ee-a413-e0869b0433b2\") " pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:21.691853 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.691822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.692022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.691863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfdnn\" (UniqueName: \"kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.792703 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.792667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.792866 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.792719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfdnn\" (UniqueName: \"kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.795257 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.795232 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.803771 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.803748 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfdnn\" (UniqueName: \"kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn\") pod \"authorino-75cdc567d8-t4whc\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:21.890005 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:21.889979 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:24:22.046049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.046026 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:24:22.047938 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:24:22.047910 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01731d2d_732e_452c_85d6_45ac00f393d6.slice/crio-92ac70fb911cc1ff6a1ae49ee20e2d7766f9efeafd9eb8e5694786431f1a2598 WatchSource:0}: Error finding container 92ac70fb911cc1ff6a1ae49ee20e2d7766f9efeafd9eb8e5694786431f1a2598: Status 404 returned error can't find the container with id 92ac70fb911cc1ff6a1ae49ee20e2d7766f9efeafd9eb8e5694786431f1a2598 Jul 06 14:24:22.535827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.535792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75cdc567d8-t4whc" event={"ID":"01731d2d-732e-452c-85d6-45ac00f393d6","Type":"ContainerStarted","Data":"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23"} Jul 06 14:24:22.536294 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.535842 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75cdc567d8-t4whc" event={"ID":"01731d2d-732e-452c-85d6-45ac00f393d6","Type":"ContainerStarted","Data":"92ac70fb911cc1ff6a1ae49ee20e2d7766f9efeafd9eb8e5694786431f1a2598"} Jul 06 14:24:22.537396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.537365 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" event={"ID":"b10f118c-6087-4d73-a2e9-33aa1d40d50b","Type":"ContainerStarted","Data":"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3"} Jul 06 14:24:22.537396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.537393 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:22.537572 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.537397 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" podUID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" containerName="authorino" containerID="cri-o://b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3" gracePeriod=30 Jul 06 14:24:22.544133 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.544111 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:22.558094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.558047 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75cdc567d8-t4whc" podStartSLOduration=1.239140573 podStartE2EDuration="1.558033735s" totalImagesPullingTime="318.893162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:22.557408265 +0000 UTC m=+578.252248511" watchObservedRunningTime="2026-07-06 14:24:22.558033735 +0000 UTC m=+578.252873981" Jul 06 14:24:22.596080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.596044 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:24:22.596279 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.596256 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-x496m" podUID="350ee84c-9a14-4250-b917-3854e99e35ad" containerName="authorino" containerID="cri-o://8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9" gracePeriod=30 Jul 06 14:24:22.605482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.605438 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" podStartSLOduration=1.25461936 podStartE2EDuration="1.605423107s" totalImagesPullingTime="350.803747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:22.60236102 +0000 UTC m=+578.297201267" watchObservedRunningTime="2026-07-06 14:24:22.605423107 +0000 UTC m=+578.300263353" Jul 06 14:24:22.699923 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.699893 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llxvc\" (UniqueName: \"kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc\") pod \"0c0bf967-a0f8-43ee-a413-e0869b0433b2\" (UID: \"0c0bf967-a0f8-43ee-a413-e0869b0433b2\") " Jul 06 14:24:22.702097 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.702069 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc" (OuterVolumeSpecName: "kube-api-access-llxvc") pod "0c0bf967-a0f8-43ee-a413-e0869b0433b2" (UID: "0c0bf967-a0f8-43ee-a413-e0869b0433b2"). InnerVolumeSpecName "kube-api-access-llxvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:22.801338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.801308 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llxvc\" (UniqueName: \"kubernetes.io/projected/0c0bf967-a0f8-43ee-a413-e0869b0433b2-kube-api-access-llxvc\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:24:22.851053 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.851028 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:22.869313 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:22.869273 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:24:23.003038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.002964 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pwdm\" (UniqueName: \"kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm\") pod \"b10f118c-6087-4d73-a2e9-33aa1d40d50b\" (UID: \"b10f118c-6087-4d73-a2e9-33aa1d40d50b\") " Jul 06 14:24:23.003232 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.003048 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9q78\" (UniqueName: \"kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78\") pod \"350ee84c-9a14-4250-b917-3854e99e35ad\" (UID: \"350ee84c-9a14-4250-b917-3854e99e35ad\") " Jul 06 14:24:23.004984 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.004960 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm" (OuterVolumeSpecName: "kube-api-access-8pwdm") pod "b10f118c-6087-4d73-a2e9-33aa1d40d50b" (UID: "b10f118c-6087-4d73-a2e9-33aa1d40d50b"). InnerVolumeSpecName "kube-api-access-8pwdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:23.005103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.005062 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78" (OuterVolumeSpecName: "kube-api-access-w9q78") pod "350ee84c-9a14-4250-b917-3854e99e35ad" (UID: "350ee84c-9a14-4250-b917-3854e99e35ad"). InnerVolumeSpecName "kube-api-access-w9q78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:23.103977 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.103938 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9q78\" (UniqueName: \"kubernetes.io/projected/350ee84c-9a14-4250-b917-3854e99e35ad-kube-api-access-w9q78\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:24:23.103977 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.103971 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pwdm\" (UniqueName: \"kubernetes.io/projected/b10f118c-6087-4d73-a2e9-33aa1d40d50b-kube-api-access-8pwdm\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:24:23.543566 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.543526 2581 generic.go:356] "Generic (PLEG): container finished" podID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" containerID="b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3" exitCode=0 Jul 06 14:24:23.544012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.543571 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" Jul 06 14:24:23.544012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.543609 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" event={"ID":"b10f118c-6087-4d73-a2e9-33aa1d40d50b","Type":"ContainerDied","Data":"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3"} Jul 06 14:24:23.544012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.543652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gs2sd" event={"ID":"b10f118c-6087-4d73-a2e9-33aa1d40d50b","Type":"ContainerDied","Data":"733a7a66aff80ff1f8b1a1607b2ae27dec1bb28f32bcab6f1420becb6a4129fa"} Jul 06 14:24:23.544012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.543670 2581 scope.go:117] "RemoveContainer" containerID="b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3" Jul 06 14:24:23.545076 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.545055 2581 generic.go:356] "Generic (PLEG): container finished" podID="350ee84c-9a14-4250-b917-3854e99e35ad" containerID="8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9" exitCode=0 Jul 06 14:24:23.545160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.545088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x496m" event={"ID":"350ee84c-9a14-4250-b917-3854e99e35ad","Type":"ContainerDied","Data":"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9"} Jul 06 14:24:23.545160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.545125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-x496m" event={"ID":"350ee84c-9a14-4250-b917-3854e99e35ad","Type":"ContainerDied","Data":"ef8abc920c5d7acb30aabac656820d039301170b3865aad91eece897632ee3ef"} Jul 06 14:24:23.545160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.545098 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-x496m" Jul 06 14:24:23.545160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.545142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-854d59684-h9zht" Jul 06 14:24:23.554418 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.554358 2581 scope.go:117] "RemoveContainer" containerID="b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3" Jul 06 14:24:23.554718 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:23.554698 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3\": container with ID starting with b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3 not found: ID does not exist" containerID="b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3" Jul 06 14:24:23.554798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.554725 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3"} err="failed to get container status \"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3\": rpc error: code = NotFound desc = could not find container \"b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3\": container with ID starting with b619d613b4b78bd9ec1730e48fc2b4798a655478dcd93de444b54f080887feb3 not found: ID does not exist" Jul 06 14:24:23.554798 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.554747 2581 scope.go:117] "RemoveContainer" containerID="8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9" Jul 06 14:24:23.562634 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.562618 2581 scope.go:117] "RemoveContainer" containerID="8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9" Jul 06 14:24:23.562850 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:23.562835 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9\": container with ID starting with 8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9 not found: ID does not exist" containerID="8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9" Jul 06 14:24:23.562894 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.562855 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9"} err="failed to get container status \"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9\": rpc error: code = NotFound desc = could not find container \"8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9\": container with ID starting with 8e69d647908eef5f026569809ccd876abf55e261730fb7b074706d74f8f00ff9 not found: ID does not exist" Jul 06 14:24:23.612966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.612931 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-854d59684-h9zht"] Jul 06 14:24:23.624557 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.624527 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-854d59684-h9zht"] Jul 06 14:24:23.644720 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.644675 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:24:23.654131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.654099 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-x496m"] Jul 06 14:24:23.682751 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.682726 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:23.695058 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:23.695029 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gs2sd"] Jul 06 14:24:24.816578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:24.816537 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c0bf967-a0f8-43ee-a413-e0869b0433b2" path="/var/lib/kubelet/pods/0c0bf967-a0f8-43ee-a413-e0869b0433b2/volumes" Jul 06 14:24:24.817073 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:24.816855 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="350ee84c-9a14-4250-b917-3854e99e35ad" path="/var/lib/kubelet/pods/350ee84c-9a14-4250-b917-3854e99e35ad/volumes" Jul 06 14:24:24.817316 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:24.817298 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" path="/var/lib/kubelet/pods/b10f118c-6087-4d73-a2e9-33aa1d40d50b/volumes" Jul 06 14:24:27.537643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.537610 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.537947 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" containerName="authorino" Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.537958 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" containerName="authorino" Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.537967 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="350ee84c-9a14-4250-b917-3854e99e35ad" containerName="authorino" Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.537973 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="350ee84c-9a14-4250-b917-3854e99e35ad" containerName="authorino" Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.538031 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="b10f118c-6087-4d73-a2e9-33aa1d40d50b" containerName="authorino" Jul 06 14:24:27.538051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.538043 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="350ee84c-9a14-4250-b917-3854e99e35ad" containerName="authorino" Jul 06 14:24:27.541098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.541077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.543511 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.543491 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.543606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.543535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dttfc\" (UniqueName: \"kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.546160 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.546141 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:24:27.547056 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.547025 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5qbf2\"" Jul 06 14:24:27.553615 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.553589 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:24:27.559132 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.559095 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:27.644123 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.644087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dttfc\" (UniqueName: \"kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.644312 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.644162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.644312 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:27.644290 2581 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 14:24:27.644391 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:27.644364 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert podName:12ff4ce0-0154-49c3-add2-0ca07d48e3a2 nodeName:}" failed. No retries permitted until 2026-07-06 14:24:28.14434711 +0000 UTC m=+583.839187335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert") pod "maas-controller-864465c5bf-c6nl5" (UID: "12ff4ce0-0154-49c3-add2-0ca07d48e3a2") : secret "maas-controller-webhook-cert" not found Jul 06 14:24:27.656766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.656739 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dttfc\" (UniqueName: \"kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:27.744806 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.744768 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:24:27.747062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.747046 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.769595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.769568 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:24:27.845337 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.845253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.845485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.845412 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfxdx\" (UniqueName: \"kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.946251 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.946187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfxdx\" (UniqueName: \"kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.946413 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.946278 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.948796 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.948776 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:27.958970 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:27.958941 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfxdx\" (UniqueName: \"kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx\") pod \"maas-controller-bf594b9b9-zs8ms\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:28.057062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.057029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:28.147858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.147824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:28.150718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.150686 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") pod \"maas-controller-864465c5bf-c6nl5\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:28.158941 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.158912 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:28.215299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.215098 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:24:28.216251 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:24:28.216187 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bc0851_2c84_44c9_b940_f2ab1e2f5afa.slice/crio-d64cf507281ff8b5b3d170518f6944724c27724d9202e074cd5d5fce5885e617 WatchSource:0}: Error finding container d64cf507281ff8b5b3d170518f6944724c27724d9202e074cd5d5fce5885e617: Status 404 returned error can't find the container with id d64cf507281ff8b5b3d170518f6944724c27724d9202e074cd5d5fce5885e617 Jul 06 14:24:28.307420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.307392 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:28.307606 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:24:28.307585 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ff4ce0_0154_49c3_add2_0ca07d48e3a2.slice/crio-6fb25d1d48a92e3ac8b591b476db2a07c0481ade86a0cf72d63228ddeac82fc1 WatchSource:0}: Error finding container 6fb25d1d48a92e3ac8b591b476db2a07c0481ade86a0cf72d63228ddeac82fc1: Status 404 returned error can't find the container with id 6fb25d1d48a92e3ac8b591b476db2a07c0481ade86a0cf72d63228ddeac82fc1 Jul 06 14:24:28.573955 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.573925 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864465c5bf-c6nl5" event={"ID":"12ff4ce0-0154-49c3-add2-0ca07d48e3a2","Type":"ContainerStarted","Data":"6fb25d1d48a92e3ac8b591b476db2a07c0481ade86a0cf72d63228ddeac82fc1"} Jul 06 14:24:28.575157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:28.575137 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" event={"ID":"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa","Type":"ContainerStarted","Data":"d64cf507281ff8b5b3d170518f6944724c27724d9202e074cd5d5fce5885e617"} Jul 06 14:24:31.596084 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.596040 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864465c5bf-c6nl5" event={"ID":"12ff4ce0-0154-49c3-add2-0ca07d48e3a2","Type":"ContainerStarted","Data":"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18"} Jul 06 14:24:31.596581 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.596165 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:31.597887 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.597864 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" event={"ID":"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa","Type":"ContainerStarted","Data":"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99"} Jul 06 14:24:31.598010 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.597917 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:31.642028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.641974 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-864465c5bf-c6nl5" podStartSLOduration=2.33630425 podStartE2EDuration="4.641953066s" totalImagesPullingTime="2.305648816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:31.640188641 +0000 UTC m=+587.335028907" watchObservedRunningTime="2026-07-06 14:24:31.641953066 +0000 UTC m=+587.336793315" Jul 06 14:24:31.681031 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:31.680974 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" podStartSLOduration=2.294767837 podStartE2EDuration="4.680958918s" totalImagesPullingTime="2.386191081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:24:31.678795194 +0000 UTC m=+587.373635441" watchObservedRunningTime="2026-07-06 14:24:31.680958918 +0000 UTC m=+587.375799165" Jul 06 14:24:42.616627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:42.616589 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:42.616991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:42.616658 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:24:42.762024 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:42.761984 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:42.762267 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:42.762241 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-864465c5bf-c6nl5" podUID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" containerName="manager" containerID="cri-o://4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18" gracePeriod=30 Jul 06 14:24:43.013752 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.013727 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:43.076987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.076959 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dttfc\" (UniqueName: \"kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc\") pod \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " Jul 06 14:24:43.077139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.077009 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") pod \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\" (UID: \"12ff4ce0-0154-49c3-add2-0ca07d48e3a2\") " Jul 06 14:24:43.079351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.079318 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "12ff4ce0-0154-49c3-add2-0ca07d48e3a2" (UID: "12ff4ce0-0154-49c3-add2-0ca07d48e3a2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:24:43.079474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.079426 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc" (OuterVolumeSpecName: "kube-api-access-dttfc") pod "12ff4ce0-0154-49c3-add2-0ca07d48e3a2" (UID: "12ff4ce0-0154-49c3-add2-0ca07d48e3a2"). InnerVolumeSpecName "kube-api-access-dttfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:24:43.178431 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.178358 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dttfc\" (UniqueName: \"kubernetes.io/projected/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-kube-api-access-dttfc\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:24:43.178431 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.178385 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12ff4ce0-0154-49c3-add2-0ca07d48e3a2-webhook-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:24:43.668680 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.668640 2581 generic.go:356] "Generic (PLEG): container finished" podID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" containerID="4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18" exitCode=0 Jul 06 14:24:43.668680 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.668684 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864465c5bf-c6nl5" event={"ID":"12ff4ce0-0154-49c3-add2-0ca07d48e3a2","Type":"ContainerDied","Data":"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18"} Jul 06 14:24:43.669170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.668699 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-864465c5bf-c6nl5" Jul 06 14:24:43.669170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.668765 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-864465c5bf-c6nl5" event={"ID":"12ff4ce0-0154-49c3-add2-0ca07d48e3a2","Type":"ContainerDied","Data":"6fb25d1d48a92e3ac8b591b476db2a07c0481ade86a0cf72d63228ddeac82fc1"} Jul 06 14:24:43.669170 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.668798 2581 scope.go:117] "RemoveContainer" containerID="4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18" Jul 06 14:24:43.678368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.678350 2581 scope.go:117] "RemoveContainer" containerID="4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18" Jul 06 14:24:43.678642 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:24:43.678622 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18\": container with ID starting with 4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18 not found: ID does not exist" containerID="4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18" Jul 06 14:24:43.678691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.678653 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18"} err="failed to get container status \"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18\": rpc error: code = NotFound desc = could not find container \"4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18\": container with ID starting with 4f15964c37586823e445d1173d60d38d6df7fbc21e5af719035f09d4c8c3be18 not found: ID does not exist" Jul 06 14:24:43.699303 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.699282 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:43.703129 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:43.703108 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-864465c5bf-c6nl5"] Jul 06 14:24:44.691380 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:44.691354 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:24:44.691734 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:44.691423 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:24:44.814753 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:24:44.814722 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" path="/var/lib/kubelet/pods/12ff4ce0-0154-49c3-add2-0ca07d48e3a2/volumes" Jul 06 14:25:03.982567 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.982532 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw"] Jul 06 14:25:03.984890 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.982905 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" containerName="manager" Jul 06 14:25:03.984890 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.982916 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" containerName="manager" Jul 06 14:25:03.984890 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.982986 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ff4ce0-0154-49c3-add2-0ca07d48e3a2" containerName="manager" Jul 06 14:25:03.985437 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.985177 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:03.989746 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.989727 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cq7pn\"" Jul 06 14:25:03.989870 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:03.989768 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 14:25:04.016137 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.016103 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw"] Jul 06 14:25:04.063927 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.063890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.064096 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.063943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q29\" (UniqueName: \"kubernetes.io/projected/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-kube-api-access-r8q29\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.111102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.111064 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-4tsqt"] Jul 06 14:25:04.113699 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.113681 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.135940 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.135910 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-4tsqt"] Jul 06 14:25:04.164694 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.164669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q29\" (UniqueName: \"kubernetes.io/projected/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-kube-api-access-r8q29\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.164846 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.164748 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.165470 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.165438 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.196897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.196865 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q29\" (UniqueName: \"kubernetes.io/projected/8332bf2d-7242-4bcf-86ec-ec6ebe412f80-kube-api-access-r8q29\") pod \"payload-pre-processing-59f49fd4f6-bp4mw\" (UID: \"8332bf2d-7242-4bcf-86ec-ec6ebe412f80\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.265990 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.265898 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8141b573-fcf4-4afe-af80-b06de3bc027a-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.265990 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.265963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6vw4\" (UniqueName: \"kubernetes.io/projected/8141b573-fcf4-4afe-af80-b06de3bc027a-kube-api-access-z6vw4\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.296005 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.295966 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:04.367590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.367555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8141b573-fcf4-4afe-af80-b06de3bc027a-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.367744 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.367640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6vw4\" (UniqueName: \"kubernetes.io/projected/8141b573-fcf4-4afe-af80-b06de3bc027a-kube-api-access-z6vw4\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.368263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.368243 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8141b573-fcf4-4afe-af80-b06de3bc027a-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.408578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.408544 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6vw4\" (UniqueName: \"kubernetes.io/projected/8141b573-fcf4-4afe-af80-b06de3bc027a-kube-api-access-z6vw4\") pod \"payload-processing-7bbcb654f4-4tsqt\" (UID: \"8141b573-fcf4-4afe-af80-b06de3bc027a\") " pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.423365 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.423340 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:04.504131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.504110 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw"] Jul 06 14:25:04.506672 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:25:04.506573 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8332bf2d_7242_4bcf_86ec_ec6ebe412f80.slice/crio-d57ed7cb0af7628f77e0f99d41273354bb3a9612b7cad39ae6f85f4995faca3a WatchSource:0}: Error finding container d57ed7cb0af7628f77e0f99d41273354bb3a9612b7cad39ae6f85f4995faca3a: Status 404 returned error can't find the container with id d57ed7cb0af7628f77e0f99d41273354bb3a9612b7cad39ae6f85f4995faca3a Jul 06 14:25:04.509122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.509099 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 14:25:04.596067 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.596042 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-4tsqt"] Jul 06 14:25:04.597973 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:25:04.597947 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8141b573_fcf4_4afe_af80_b06de3bc027a.slice/crio-d49ea6d6f4c061336d467e31d2fef0355e2e1a090f01bad56b7654b5f17b4424 WatchSource:0}: Error finding container d49ea6d6f4c061336d467e31d2fef0355e2e1a090f01bad56b7654b5f17b4424: Status 404 returned error can't find the container with id d49ea6d6f4c061336d467e31d2fef0355e2e1a090f01bad56b7654b5f17b4424 Jul 06 14:25:04.782112 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.782078 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" event={"ID":"8141b573-fcf4-4afe-af80-b06de3bc027a","Type":"ContainerStarted","Data":"d49ea6d6f4c061336d467e31d2fef0355e2e1a090f01bad56b7654b5f17b4424"} Jul 06 14:25:04.783188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:04.783167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" event={"ID":"8332bf2d-7242-4bcf-86ec-ec6ebe412f80","Type":"ContainerStarted","Data":"d57ed7cb0af7628f77e0f99d41273354bb3a9612b7cad39ae6f85f4995faca3a"} Jul 06 14:25:08.838461 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.838414 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" event={"ID":"8332bf2d-7242-4bcf-86ec-ec6ebe412f80","Type":"ContainerStarted","Data":"e71019456146d147aca07bec0476c462e9624ac2cf70f1e302474d83f7c99d6a"} Jul 06 14:25:08.838938 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.838513 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:08.840169 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.840142 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" event={"ID":"8141b573-fcf4-4afe-af80-b06de3bc027a","Type":"ContainerStarted","Data":"c5f4aba2590dd5dea73e6160fcf66150a6337473b8d47e653bdf4daf7bff97f9"} Jul 06 14:25:08.840329 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.840267 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:08.864322 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.864266 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" podStartSLOduration=2.623598081 podStartE2EDuration="5.86425142s" totalImagesPullingTime="3.240653339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:08.862424252 +0000 UTC m=+624.557264498" watchObservedRunningTime="2026-07-06 14:25:08.86425142 +0000 UTC m=+624.559091692" Jul 06 14:25:08.894511 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:08.894466 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" podStartSLOduration=1.734768824 podStartE2EDuration="4.894452055s" totalImagesPullingTime="3.159683231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:08.893551002 +0000 UTC m=+624.588391249" watchObservedRunningTime="2026-07-06 14:25:08.894452055 +0000 UTC m=+624.589292302" Jul 06 14:25:11.859383 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.859351 2581 generic.go:356] "Generic (PLEG): container finished" podID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" containerID="b13ae3d934880a8b19cc64b5a81fa38d8ecf7bdc744f515115f0cbe4ed519494" exitCode=0 Jul 06 14:25:11.859790 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.859426 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" event={"ID":"37bf8f08-e9ec-4541-ad03-b20618f0a6fd","Type":"ContainerDied","Data":"b13ae3d934880a8b19cc64b5a81fa38d8ecf7bdc744f515115f0cbe4ed519494"} Jul 06 14:25:11.859790 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.859780 2581 scope.go:117] "RemoveContainer" containerID="b13ae3d934880a8b19cc64b5a81fa38d8ecf7bdc744f515115f0cbe4ed519494" Jul 06 14:25:11.860779 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.860754 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:25:11.860840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.860824 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:25:11.860882 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:11.860857 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:25:12.866966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:12.866933 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" event={"ID":"37bf8f08-e9ec-4541-ad03-b20618f0a6fd","Type":"ContainerStarted","Data":"9dac8ed4538e7c52b303f8c9470d5be436118a6ac0ab80abe10a78a2b3b972f4"} Jul 06 14:25:13.041383 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:13.041344 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:25:13.042377 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:13.042352 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-psc62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 06 14:25:13.042484 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:13.042409 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" podUID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 06 14:25:13.940305 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:13.940272 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:25:14.040778 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:14.040738 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-psc62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 06 14:25:14.040959 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:14.040794 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" podUID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 06 14:25:15.041551 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:15.041513 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-psc62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 06 14:25:15.042028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:15.041581 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" podUID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 06 14:25:15.385589 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:15.385501 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:15.385759 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:15.385724 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" containerID="cri-o://296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e" gracePeriod=30 Jul 06 14:25:16.041019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:16.040978 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-psc62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 06 14:25:16.041284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:16.041053 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" podUID="37bf8f08-e9ec-4541-ad03-b20618f0a6fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 06 14:25:17.051435 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.051410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:25:17.051922 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.051902 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:17.052531 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.052515 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-psc62" Jul 06 14:25:17.201703 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.201681 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tz62\" (UniqueName: \"kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62\") pod \"258e6da1-d347-48c7-8856-504e9152529b\" (UID: \"258e6da1-d347-48c7-8856-504e9152529b\") " Jul 06 14:25:17.204280 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.204247 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62" (OuterVolumeSpecName: "kube-api-access-8tz62") pod "258e6da1-d347-48c7-8856-504e9152529b" (UID: "258e6da1-d347-48c7-8856-504e9152529b"). InnerVolumeSpecName "kube-api-access-8tz62". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:25:17.302323 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.302290 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tz62\" (UniqueName: \"kubernetes.io/projected/258e6da1-d347-48c7-8856-504e9152529b-kube-api-access-8tz62\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:25:17.893784 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.893752 2581 generic.go:356] "Generic (PLEG): container finished" podID="258e6da1-d347-48c7-8856-504e9152529b" containerID="296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e" exitCode=143 Jul 06 14:25:17.893983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.893791 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:17.893983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.893839 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"258e6da1-d347-48c7-8856-504e9152529b","Type":"ContainerDied","Data":"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e"} Jul 06 14:25:17.893983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.893883 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"258e6da1-d347-48c7-8856-504e9152529b","Type":"ContainerDied","Data":"3d92a5cd44ed1df4ed3b0e636f6f26cdb8bea9996ed59fe92655dfa921cc0f4c"} Jul 06 14:25:17.893983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.893904 2581 scope.go:117] "RemoveContainer" containerID="296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e" Jul 06 14:25:17.904054 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.904035 2581 scope.go:117] "RemoveContainer" containerID="296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e" Jul 06 14:25:17.904332 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:25:17.904310 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e\": container with ID starting with 296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e not found: ID does not exist" containerID="296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e" Jul 06 14:25:17.904398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.904342 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e"} err="failed to get container status \"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e\": rpc error: code = NotFound desc = could not find container \"296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e\": container with ID starting with 296af437e64a8e660b2d08aaa0e43d39152b57d3b1a5f0c680745c5156c4c60e not found: ID does not exist" Jul 06 14:25:17.923032 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.923011 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:17.933696 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.933674 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:17.971292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.971263 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:17.971659 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.971643 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" Jul 06 14:25:17.971747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.971662 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" Jul 06 14:25:17.971802 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.971775 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="258e6da1-d347-48c7-8856-504e9152529b" containerName="keycloak" Jul 06 14:25:17.974465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.974447 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:17.978956 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.978936 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 06 14:25:17.979258 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.979239 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 06 14:25:17.979309 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.979299 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 06 14:25:17.979351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.979251 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 06 14:25:17.980343 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.980323 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-6j574\"" Jul 06 14:25:17.990537 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:17.990514 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:18.006691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.006668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0ae14780-74f8-4188-9d05-4e56b9096d39-test-realms\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.006793 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.006736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnprp\" (UniqueName: \"kubernetes.io/projected/0ae14780-74f8-4188-9d05-4e56b9096d39-kube-api-access-rnprp\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.107777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.107743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnprp\" (UniqueName: \"kubernetes.io/projected/0ae14780-74f8-4188-9d05-4e56b9096d39-kube-api-access-rnprp\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.108155 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.107799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0ae14780-74f8-4188-9d05-4e56b9096d39-test-realms\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.108477 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.108459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/0ae14780-74f8-4188-9d05-4e56b9096d39-test-realms\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.130331 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.130306 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnprp\" (UniqueName: \"kubernetes.io/projected/0ae14780-74f8-4188-9d05-4e56b9096d39-kube-api-access-rnprp\") pod \"maas-keycloak-0\" (UID: \"0ae14780-74f8-4188-9d05-4e56b9096d39\") " pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.283647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.283616 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:18.422570 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.422547 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 06 14:25:18.424387 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:25:18.424361 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae14780_74f8_4188_9d05_4e56b9096d39.slice/crio-0f5f3c12fd225e5a93b4cf8d683ed9ef4a4ca4035ea5f3d4643e1d6c8f543098 WatchSource:0}: Error finding container 0f5f3c12fd225e5a93b4cf8d683ed9ef4a4ca4035ea5f3d4643e1d6c8f543098: Status 404 returned error can't find the container with id 0f5f3c12fd225e5a93b4cf8d683ed9ef4a4ca4035ea5f3d4643e1d6c8f543098 Jul 06 14:25:18.814413 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.814380 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="258e6da1-d347-48c7-8856-504e9152529b" path="/var/lib/kubelet/pods/258e6da1-d347-48c7-8856-504e9152529b/volumes" Jul 06 14:25:18.901299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:18.901263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0ae14780-74f8-4188-9d05-4e56b9096d39","Type":"ContainerStarted","Data":"0f5f3c12fd225e5a93b4cf8d683ed9ef4a4ca4035ea5f3d4643e1d6c8f543098"} Jul 06 14:25:19.849142 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:19.849109 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-bp4mw" Jul 06 14:25:19.849628 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:19.849221 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-4tsqt" Jul 06 14:25:19.911119 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:19.911084 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"0ae14780-74f8-4188-9d05-4e56b9096d39","Type":"ContainerStarted","Data":"30607000d84d7486835d6cd11917b02e75850f32fe965f8ce0bf6c2788b4ae00"} Jul 06 14:25:20.285808 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:20.285774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:20.286075 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:20.285829 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:21.285158 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:21.285100 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:22.284745 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:22.284693 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:23.284446 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:23.284401 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:24.284574 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:24.284529 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:25.284855 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:25.284803 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:26.284519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:26.284466 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:27.284782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:27.284728 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:28.284842 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:28.284544 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:28.285362 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:28.284844 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:29.284220 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:29.284130 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:30.284352 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:30.284299 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:31.284416 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:31.284366 2581 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.53:9000/health/started\": dial tcp 10.132.0.53:9000: connect: connection refused" Jul 06 14:25:32.392737 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:32.392696 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:32.406299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:32.406235 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="0ae14780-74f8-4188-9d05-4e56b9096d39" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:25:32.422029 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:32.421956 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=14.880092803 podStartE2EDuration="15.421936854s" totalImagesPullingTime="541.844051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:19.964176033 +0000 UTC m=+635.659016277" watchObservedRunningTime="2026-07-06 14:25:32.421936854 +0000 UTC m=+648.116777101" Jul 06 14:25:42.399355 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:42.399324 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 06 14:25:53.209727 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.209694 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:25:53.215588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.215567 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.219612 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.219580 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jul 06 14:25:53.240282 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.240253 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:25:53.330184 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.330147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.330382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.330213 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.330428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.330384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfw6\" (UniqueName: \"kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.431429 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.431385 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.431596 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.431438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.431596 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.431469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfw6\" (UniqueName: \"kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.432116 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.432096 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.433982 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.433957 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.448983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.448956 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfw6\" (UniqueName: \"kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6\") pod \"authorino-66975db9cd-rj5xw\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.525059 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.524977 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:25:53.692575 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:25:53.692544 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5535a0c9_ebc9_4200_98e3_7e2ea02b0052.slice/crio-0e734f1c306e6da6ee6dfe6b6ede2d81cf07e2e0acf27613db371b39798f677a WatchSource:0}: Error finding container 0e734f1c306e6da6ee6dfe6b6ede2d81cf07e2e0acf27613db371b39798f677a: Status 404 returned error can't find the container with id 0e734f1c306e6da6ee6dfe6b6ede2d81cf07e2e0acf27613db371b39798f677a Jul 06 14:25:53.693444 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:53.693418 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:25:54.135749 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:54.135719 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66975db9cd-rj5xw" event={"ID":"5535a0c9-ebc9-4200-98e3-7e2ea02b0052","Type":"ContainerStarted","Data":"0e734f1c306e6da6ee6dfe6b6ede2d81cf07e2e0acf27613db371b39798f677a"} Jul 06 14:25:55.142367 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.142335 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66975db9cd-rj5xw" event={"ID":"5535a0c9-ebc9-4200-98e3-7e2ea02b0052","Type":"ContainerStarted","Data":"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb"} Jul 06 14:25:55.170737 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.170688 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66975db9cd-rj5xw" podStartSLOduration=1.793269698 podStartE2EDuration="2.170671606s" totalImagesPullingTime="377.401908ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:25:55.167893819 +0000 UTC m=+670.862734063" watchObservedRunningTime="2026-07-06 14:25:55.170671606 +0000 UTC m=+670.865511852" Jul 06 14:25:55.215807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.215774 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:25:55.215997 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.215972 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75cdc567d8-t4whc" podUID="01731d2d-732e-452c-85d6-45ac00f393d6" containerName="authorino" containerID="cri-o://063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23" gracePeriod=30 Jul 06 14:25:55.459797 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.459768 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:25:55.552467 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.552431 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert\") pod \"01731d2d-732e-452c-85d6-45ac00f393d6\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " Jul 06 14:25:55.552642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.552579 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfdnn\" (UniqueName: \"kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn\") pod \"01731d2d-732e-452c-85d6-45ac00f393d6\" (UID: \"01731d2d-732e-452c-85d6-45ac00f393d6\") " Jul 06 14:25:55.554719 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.554684 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn" (OuterVolumeSpecName: "kube-api-access-cfdnn") pod "01731d2d-732e-452c-85d6-45ac00f393d6" (UID: "01731d2d-732e-452c-85d6-45ac00f393d6"). InnerVolumeSpecName "kube-api-access-cfdnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:25:55.563945 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.563911 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "01731d2d-732e-452c-85d6-45ac00f393d6" (UID: "01731d2d-732e-452c-85d6-45ac00f393d6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:25:55.653996 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.653956 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfdnn\" (UniqueName: \"kubernetes.io/projected/01731d2d-732e-452c-85d6-45ac00f393d6-kube-api-access-cfdnn\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:25:55.653996 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:55.653988 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/01731d2d-732e-452c-85d6-45ac00f393d6-tls-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:25:56.149050 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.149017 2581 generic.go:356] "Generic (PLEG): container finished" podID="01731d2d-732e-452c-85d6-45ac00f393d6" containerID="063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23" exitCode=0 Jul 06 14:25:56.149485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.149062 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75cdc567d8-t4whc" Jul 06 14:25:56.149485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.149106 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75cdc567d8-t4whc" event={"ID":"01731d2d-732e-452c-85d6-45ac00f393d6","Type":"ContainerDied","Data":"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23"} Jul 06 14:25:56.149485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.149152 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75cdc567d8-t4whc" event={"ID":"01731d2d-732e-452c-85d6-45ac00f393d6","Type":"ContainerDied","Data":"92ac70fb911cc1ff6a1ae49ee20e2d7766f9efeafd9eb8e5694786431f1a2598"} Jul 06 14:25:56.149485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.149175 2581 scope.go:117] "RemoveContainer" containerID="063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23" Jul 06 14:25:56.159301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.159282 2581 scope.go:117] "RemoveContainer" containerID="063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23" Jul 06 14:25:56.159570 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:25:56.159552 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23\": container with ID starting with 063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23 not found: ID does not exist" containerID="063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23" Jul 06 14:25:56.159639 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.159582 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23"} err="failed to get container status \"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23\": rpc error: code = NotFound desc = could not find container \"063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23\": container with ID starting with 063fb7cac36605227aed58df116e085198a90b96e1192a3fa356b6764b73ed23 not found: ID does not exist" Jul 06 14:25:56.254640 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.254607 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:25:56.257482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.257454 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75cdc567d8-t4whc"] Jul 06 14:25:56.815329 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:25:56.815296 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01731d2d-732e-452c-85d6-45ac00f393d6" path="/var/lib/kubelet/pods/01731d2d-732e-452c-85d6-45ac00f393d6/volumes" Jul 06 14:26:17.766858 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.766811 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7"] Jul 06 14:26:17.767451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.767425 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01731d2d-732e-452c-85d6-45ac00f393d6" containerName="authorino" Jul 06 14:26:17.767451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.767451 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="01731d2d-732e-452c-85d6-45ac00f393d6" containerName="authorino" Jul 06 14:26:17.767607 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.767551 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="01731d2d-732e-452c-85d6-45ac00f393d6" containerName="authorino" Jul 06 14:26:17.771070 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.771045 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.780427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.780404 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lrj5c\"" Jul 06 14:26:17.780427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.780420 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 14:26:17.780619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.780449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 14:26:17.780619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.780419 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 14:26:17.790149 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.790124 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7"] Jul 06 14:26:17.843745 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.843693 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.843944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.843761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tf9\" (UniqueName: \"kubernetes.io/projected/4e7bba0c-3e23-4a6d-b807-3d80d2082118-kube-api-access-k7tf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.843944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.843857 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.843944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.843901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.844114 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.843983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.844114 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.844017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944767 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tf9\" (UniqueName: \"kubernetes.io/projected/4e7bba0c-3e23-4a6d-b807-3d80d2082118-kube-api-access-k7tf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944849 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.944972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.944912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.945574 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.945544 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.945574 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.945561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.945747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.945623 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.948044 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.948023 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e7bba0c-3e23-4a6d-b807-3d80d2082118-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.948213 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.948179 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e7bba0c-3e23-4a6d-b807-3d80d2082118-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:17.954885 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:17.954863 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tf9\" (UniqueName: \"kubernetes.io/projected/4e7bba0c-3e23-4a6d-b807-3d80d2082118-kube-api-access-k7tf9\") pod \"e2e-distinct-simulated-kserve-59975d69f9-p8nx7\" (UID: \"4e7bba0c-3e23-4a6d-b807-3d80d2082118\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:18.081998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:18.081902 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:18.227914 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:18.227886 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7"] Jul 06 14:26:18.229113 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:26:18.229071 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7bba0c_3e23_4a6d_b807_3d80d2082118.slice/crio-2a50ceaf03d85604dbf902f6b9d702ee0eb82440d6064bb37d8aa12037a0bd61 WatchSource:0}: Error finding container 2a50ceaf03d85604dbf902f6b9d702ee0eb82440d6064bb37d8aa12037a0bd61: Status 404 returned error can't find the container with id 2a50ceaf03d85604dbf902f6b9d702ee0eb82440d6064bb37d8aa12037a0bd61 Jul 06 14:26:18.272467 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:18.272435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" event={"ID":"4e7bba0c-3e23-4a6d-b807-3d80d2082118","Type":"ContainerStarted","Data":"2a50ceaf03d85604dbf902f6b9d702ee0eb82440d6064bb37d8aa12037a0bd61"} Jul 06 14:26:20.285616 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:20.285583 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" event={"ID":"4e7bba0c-3e23-4a6d-b807-3d80d2082118","Type":"ContainerStarted","Data":"52f2c33f718be786454820556fc796b1ebff21286ed9f09501ee2c319908c49a"} Jul 06 14:26:20.316340 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:20.316283 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" podStartSLOduration=2.063038013 podStartE2EDuration="3.316266171s" totalImagesPullingTime="1.253228158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:20.315445247 +0000 UTC m=+696.010285613" watchObservedRunningTime="2026-07-06 14:26:20.316266171 +0000 UTC m=+696.011106417" Jul 06 14:26:28.083088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:28.083050 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:28.083510 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:28.083127 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:28.095560 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:28.095534 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:28.347448 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:28.347363 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-p8nx7" Jul 06 14:26:39.582284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.582169 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc"] Jul 06 14:26:39.585966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.585942 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.590222 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.590179 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 14:26:39.610475 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.610436 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc"] Jul 06 14:26:39.737647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737606 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.737647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737647 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.737884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf7zm\" (UniqueName: \"kubernetes.io/projected/2b9694f7-17de-4760-99f0-742835736b25-kube-api-access-rf7zm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.737884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.737884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b9694f7-17de-4760-99f0-742835736b25-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.737884 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.737812 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839093 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.838998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b9694f7-17de-4760-99f0-742835736b25-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839093 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839045 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839118 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839141 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839185 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf7zm\" (UniqueName: \"kubernetes.io/projected/2b9694f7-17de-4760-99f0-742835736b25-kube-api-access-rf7zm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839616 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839584 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839674 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.839724 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.839652 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.841653 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.841625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b9694f7-17de-4760-99f0-742835736b25-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.841653 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.841642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b9694f7-17de-4760-99f0-742835736b25-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.866828 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.866798 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf7zm\" (UniqueName: \"kubernetes.io/projected/2b9694f7-17de-4760-99f0-742835736b25-kube-api-access-rf7zm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-kvrvc\" (UID: \"2b9694f7-17de-4760-99f0-742835736b25\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:39.896760 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:39.896730 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:40.060587 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:40.060550 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc"] Jul 06 14:26:40.062272 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:26:40.062245 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b9694f7_17de_4760_99f0_742835736b25.slice/crio-2aaeca5846b1d0b474a5d05095fcccaf69dd59b69e9dfab8ef85e162f44d1254 WatchSource:0}: Error finding container 2aaeca5846b1d0b474a5d05095fcccaf69dd59b69e9dfab8ef85e162f44d1254: Status 404 returned error can't find the container with id 2aaeca5846b1d0b474a5d05095fcccaf69dd59b69e9dfab8ef85e162f44d1254 Jul 06 14:26:40.407659 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:40.407582 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" event={"ID":"2b9694f7-17de-4760-99f0-742835736b25","Type":"ContainerStarted","Data":"c6ce955a67b45e8f6b11246c204c1a779869e5044604088404f796ab77b030b2"} Jul 06 14:26:40.407838 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:40.407679 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" event={"ID":"2b9694f7-17de-4760-99f0-742835736b25","Type":"ContainerStarted","Data":"2aaeca5846b1d0b474a5d05095fcccaf69dd59b69e9dfab8ef85e162f44d1254"} Jul 06 14:26:49.897022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:49.896978 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:49.897501 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:49.897045 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:49.910031 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:49.909997 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:26:49.950083 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:49.950026 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" podStartSLOduration=10.732247823 podStartE2EDuration="10.950009013s" totalImagesPullingTime="217.76119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:26:40.450865988 +0000 UTC m=+716.145706236" watchObservedRunningTime="2026-07-06 14:26:49.950009013 +0000 UTC m=+725.644849258" Jul 06 14:26:50.477046 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:26:50.477016 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-kvrvc" Jul 06 14:28:13.885515 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.885438 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-758bdf5c95-5bw5z"] Jul 06 14:28:13.888770 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.888753 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:13.911405 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.911369 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758bdf5c95-5bw5z"] Jul 06 14:28:13.994993 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.994951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f8aa6748-8552-4167-89e6-1591d35c3c2d-tls-cert\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:13.994993 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.994980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f8aa6748-8552-4167-89e6-1591d35c3c2d-oidc-ca\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:13.995262 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:13.995021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29sg6\" (UniqueName: \"kubernetes.io/projected/f8aa6748-8552-4167-89e6-1591d35c3c2d-kube-api-access-29sg6\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.095709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.095665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29sg6\" (UniqueName: \"kubernetes.io/projected/f8aa6748-8552-4167-89e6-1591d35c3c2d-kube-api-access-29sg6\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.095905 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.095738 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f8aa6748-8552-4167-89e6-1591d35c3c2d-tls-cert\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.095905 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.095760 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f8aa6748-8552-4167-89e6-1591d35c3c2d-oidc-ca\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.096441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.096422 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/f8aa6748-8552-4167-89e6-1591d35c3c2d-oidc-ca\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.098518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.098495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f8aa6748-8552-4167-89e6-1591d35c3c2d-tls-cert\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.112528 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.112504 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29sg6\" (UniqueName: \"kubernetes.io/projected/f8aa6748-8552-4167-89e6-1591d35c3c2d-kube-api-access-29sg6\") pod \"authorino-758bdf5c95-5bw5z\" (UID: \"f8aa6748-8552-4167-89e6-1591d35c3c2d\") " pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.198542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.198506 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-758bdf5c95-5bw5z" Jul 06 14:28:14.364635 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.364245 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-758bdf5c95-5bw5z"] Jul 06 14:28:14.365819 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:28:14.365754 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf8aa6748_8552_4167_89e6_1591d35c3c2d.slice/crio-22f13f178bdf152bffeb6242f20d0f4a1d3e2a542d0e270fea20ec25a4a51eb3 WatchSource:0}: Error finding container 22f13f178bdf152bffeb6242f20d0f4a1d3e2a542d0e270fea20ec25a4a51eb3: Status 404 returned error can't find the container with id 22f13f178bdf152bffeb6242f20d0f4a1d3e2a542d0e270fea20ec25a4a51eb3 Jul 06 14:28:14.935863 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.935825 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758bdf5c95-5bw5z" event={"ID":"f8aa6748-8552-4167-89e6-1591d35c3c2d","Type":"ContainerStarted","Data":"dbfdef57c48e9191932c8b878f0d559066b6679b4b6ea3a2363d35d48e609702"} Jul 06 14:28:14.936498 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.935878 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-758bdf5c95-5bw5z" event={"ID":"f8aa6748-8552-4167-89e6-1591d35c3c2d","Type":"ContainerStarted","Data":"22f13f178bdf152bffeb6242f20d0f4a1d3e2a542d0e270fea20ec25a4a51eb3"} Jul 06 14:28:14.998989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:14.998923 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-758bdf5c95-5bw5z" podStartSLOduration=1.611926946 podStartE2EDuration="1.998900378s" totalImagesPullingTime="386.973432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:28:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:28:14.985147535 +0000 UTC m=+810.679987781" watchObservedRunningTime="2026-07-06 14:28:14.998900378 +0000 UTC m=+810.693740626" Jul 06 14:28:15.177323 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.177236 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:28:15.177457 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.177412 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-66975db9cd-rj5xw" podUID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" containerName="authorino" containerID="cri-o://c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb" gracePeriod=30 Jul 06 14:28:15.449291 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.448943 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:28:15.512272 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.512241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert\") pod \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " Jul 06 14:28:15.512471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.512369 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzfw6\" (UniqueName: \"kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6\") pod \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " Jul 06 14:28:15.512471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.512392 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca\") pod \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\" (UID: \"5535a0c9-ebc9-4200-98e3-7e2ea02b0052\") " Jul 06 14:28:15.514837 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.514792 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6" (OuterVolumeSpecName: "kube-api-access-qzfw6") pod "5535a0c9-ebc9-4200-98e3-7e2ea02b0052" (UID: "5535a0c9-ebc9-4200-98e3-7e2ea02b0052"). InnerVolumeSpecName "kube-api-access-qzfw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:28:15.518370 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.518336 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "5535a0c9-ebc9-4200-98e3-7e2ea02b0052" (UID: "5535a0c9-ebc9-4200-98e3-7e2ea02b0052"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:28:15.526180 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.526151 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5535a0c9-ebc9-4200-98e3-7e2ea02b0052" (UID: "5535a0c9-ebc9-4200-98e3-7e2ea02b0052"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:28:15.613459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.613421 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzfw6\" (UniqueName: \"kubernetes.io/projected/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-kube-api-access-qzfw6\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:28:15.613459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.613452 2581 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-oidc-ca\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:28:15.613459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.613464 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5535a0c9-ebc9-4200-98e3-7e2ea02b0052-tls-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:28:15.943575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.943539 2581 generic.go:356] "Generic (PLEG): container finished" podID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" containerID="c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb" exitCode=0 Jul 06 14:28:15.944038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.943574 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66975db9cd-rj5xw" event={"ID":"5535a0c9-ebc9-4200-98e3-7e2ea02b0052","Type":"ContainerDied","Data":"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb"} Jul 06 14:28:15.944038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.943602 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66975db9cd-rj5xw" Jul 06 14:28:15.944038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.943622 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66975db9cd-rj5xw" event={"ID":"5535a0c9-ebc9-4200-98e3-7e2ea02b0052","Type":"ContainerDied","Data":"0e734f1c306e6da6ee6dfe6b6ede2d81cf07e2e0acf27613db371b39798f677a"} Jul 06 14:28:15.944038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.943645 2581 scope.go:117] "RemoveContainer" containerID="c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb" Jul 06 14:28:15.953265 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.953242 2581 scope.go:117] "RemoveContainer" containerID="c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb" Jul 06 14:28:15.953581 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:28:15.953551 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb\": container with ID starting with c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb not found: ID does not exist" containerID="c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb" Jul 06 14:28:15.953675 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.953593 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb"} err="failed to get container status \"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb\": rpc error: code = NotFound desc = could not find container \"c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb\": container with ID starting with c79f0159999d8cef861bfa1bc2e344bb279921a56edf501ccb2a164929f9e2fb not found: ID does not exist" Jul 06 14:28:15.978266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.978221 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:28:15.987037 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:15.987017 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66975db9cd-rj5xw"] Jul 06 14:28:16.822213 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:28:16.822154 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" path="/var/lib/kubelet/pods/5535a0c9-ebc9-4200-98e3-7e2ea02b0052/volumes" Jul 06 14:29:37.974646 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:37.974567 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:29:37.975227 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:37.974799 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" podUID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" containerName="manager" containerID="cri-o://9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99" gracePeriod=30 Jul 06 14:29:38.225406 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.225349 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:29:38.278964 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.278930 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfxdx\" (UniqueName: \"kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx\") pod \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " Jul 06 14:29:38.278964 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.278963 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert\") pod \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\" (UID: \"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa\") " Jul 06 14:29:38.281313 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.281283 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx" (OuterVolumeSpecName: "kube-api-access-xfxdx") pod "a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" (UID: "a7bc0851-2c84-44c9-b940-f2ab1e2f5afa"). InnerVolumeSpecName "kube-api-access-xfxdx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:29:38.281313 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.281301 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" (UID: "a7bc0851-2c84-44c9-b940-f2ab1e2f5afa"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 14:29:38.380255 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.380221 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfxdx\" (UniqueName: \"kubernetes.io/projected/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-kube-api-access-xfxdx\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:29:38.380255 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.380247 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa-webhook-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:29:38.417133 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.417106 2581 generic.go:356] "Generic (PLEG): container finished" podID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" containerID="9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99" exitCode=0 Jul 06 14:29:38.417299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.417155 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" Jul 06 14:29:38.417299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.417185 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" event={"ID":"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa","Type":"ContainerDied","Data":"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99"} Jul 06 14:29:38.417299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.417258 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-zs8ms" event={"ID":"a7bc0851-2c84-44c9-b940-f2ab1e2f5afa","Type":"ContainerDied","Data":"d64cf507281ff8b5b3d170518f6944724c27724d9202e074cd5d5fce5885e617"} Jul 06 14:29:38.417299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.417282 2581 scope.go:117] "RemoveContainer" containerID="9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99" Jul 06 14:29:38.428726 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.428307 2581 scope.go:117] "RemoveContainer" containerID="9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99" Jul 06 14:29:38.428824 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:29:38.428802 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99\": container with ID starting with 9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99 not found: ID does not exist" containerID="9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99" Jul 06 14:29:38.428934 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.428829 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99"} err="failed to get container status \"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99\": rpc error: code = NotFound desc = could not find container \"9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99\": container with ID starting with 9adec6696892a1f1244e21691a18e7e9339956dc5dbc307f09297ab4a466aa99 not found: ID does not exist" Jul 06 14:29:38.443895 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.443870 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:29:38.448320 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.448297 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-zs8ms"] Jul 06 14:29:38.814959 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:38.814929 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" path="/var/lib/kubelet/pods/a7bc0851-2c84-44c9-b940-f2ab1e2f5afa/volumes" Jul 06 14:29:39.292544 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292511 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-bf594b9b9-jkxz9"] Jul 06 14:29:39.292907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292873 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" containerName="manager" Jul 06 14:29:39.292907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292884 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" containerName="manager" Jul 06 14:29:39.292907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292908 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" containerName="authorino" Jul 06 14:29:39.293011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292914 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" containerName="authorino" Jul 06 14:29:39.293011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292969 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="5535a0c9-ebc9-4200-98e3-7e2ea02b0052" containerName="authorino" Jul 06 14:29:39.293011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.292977 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7bc0851-2c84-44c9-b940-f2ab1e2f5afa" containerName="manager" Jul 06 14:29:39.295923 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.295900 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.300324 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.300305 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 14:29:39.300436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.300353 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5qbf2\"" Jul 06 14:29:39.301794 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.301778 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 14:29:39.316117 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.316091 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-jkxz9"] Jul 06 14:29:39.388888 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.388854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/509cda6e-f444-4596-8556-3a31a03c9da2-webhook-cert\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.389042 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.388958 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f42rn\" (UniqueName: \"kubernetes.io/projected/509cda6e-f444-4596-8556-3a31a03c9da2-kube-api-access-f42rn\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.489833 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.489792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/509cda6e-f444-4596-8556-3a31a03c9da2-webhook-cert\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.490011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.489871 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f42rn\" (UniqueName: \"kubernetes.io/projected/509cda6e-f444-4596-8556-3a31a03c9da2-kube-api-access-f42rn\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.492548 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.492524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/509cda6e-f444-4596-8556-3a31a03c9da2-webhook-cert\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.501239 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.501188 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f42rn\" (UniqueName: \"kubernetes.io/projected/509cda6e-f444-4596-8556-3a31a03c9da2-kube-api-access-f42rn\") pod \"maas-controller-bf594b9b9-jkxz9\" (UID: \"509cda6e-f444-4596-8556-3a31a03c9da2\") " pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.606274 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.606170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:39.748389 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:39.748267 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-bf594b9b9-jkxz9"] Jul 06 14:29:39.750993 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:29:39.750965 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509cda6e_f444_4596_8556_3a31a03c9da2.slice/crio-1fc9328e1ab76f673ce8fcb34d81c19746f18c2a8b7f66922e0dfbb0d56fd98a WatchSource:0}: Error finding container 1fc9328e1ab76f673ce8fcb34d81c19746f18c2a8b7f66922e0dfbb0d56fd98a: Status 404 returned error can't find the container with id 1fc9328e1ab76f673ce8fcb34d81c19746f18c2a8b7f66922e0dfbb0d56fd98a Jul 06 14:29:40.431651 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:40.431605 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" event={"ID":"509cda6e-f444-4596-8556-3a31a03c9da2","Type":"ContainerStarted","Data":"0b21910f0a939a10459d9d724f3c8cb783c5629c2bf79e3335b859897391bb97"} Jul 06 14:29:40.432022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:40.431713 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" event={"ID":"509cda6e-f444-4596-8556-3a31a03c9da2","Type":"ContainerStarted","Data":"1fc9328e1ab76f673ce8fcb34d81c19746f18c2a8b7f66922e0dfbb0d56fd98a"} Jul 06 14:29:40.432022 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:40.431747 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:29:40.462989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:40.462933 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" podStartSLOduration=0.954895711 podStartE2EDuration="1.462917898s" totalImagesPullingTime="508.022187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:29:40.458806333 +0000 UTC m=+896.153646583" watchObservedRunningTime="2026-07-06 14:29:40.462917898 +0000 UTC m=+896.157758144" Jul 06 14:29:44.735471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:44.735439 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:29:44.736014 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:44.735516 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:29:51.444342 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:29:51.444310 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-bf594b9b9-jkxz9" Jul 06 14:30:00.165578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.165545 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-dh2sw"] Jul 06 14:30:00.170268 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.170248 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:00.176176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.176155 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-xhhn7\"" Jul 06 14:30:00.190387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.190359 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-dh2sw"] Jul 06 14:30:00.274169 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.274128 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsmgc\" (UniqueName: \"kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc\") pod \"maas-api-key-cleanup-29722470-dh2sw\" (UID: \"428ced71-a72c-44e1-9c5e-3f1171e83df4\") " pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:00.375568 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.375529 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsmgc\" (UniqueName: \"kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc\") pod \"maas-api-key-cleanup-29722470-dh2sw\" (UID: \"428ced71-a72c-44e1-9c5e-3f1171e83df4\") " pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:00.387894 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.387864 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsmgc\" (UniqueName: \"kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc\") pod \"maas-api-key-cleanup-29722470-dh2sw\" (UID: \"428ced71-a72c-44e1-9c5e-3f1171e83df4\") " pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:00.480947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.480916 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:00.639018 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:00.638988 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722470-dh2sw"] Jul 06 14:30:00.641610 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:30:00.641577 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428ced71_a72c_44e1_9c5e_3f1171e83df4.slice/crio-6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f WatchSource:0}: Error finding container 6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f: Status 404 returned error can't find the container with id 6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f Jul 06 14:30:01.560341 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:01.560310 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" event={"ID":"428ced71-a72c-44e1-9c5e-3f1171e83df4","Type":"ContainerStarted","Data":"6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f"} Jul 06 14:30:05.590499 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:05.590458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" event={"ID":"428ced71-a72c-44e1-9c5e-3f1171e83df4","Type":"ContainerStarted","Data":"93a9f0c35a827a232eeac74dda0d2fc608e0e7fa6b6195ce3264fe275d71f071"} Jul 06 14:30:06.596844 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:06.596801 2581 generic.go:356] "Generic (PLEG): container finished" podID="428ced71-a72c-44e1-9c5e-3f1171e83df4" containerID="93a9f0c35a827a232eeac74dda0d2fc608e0e7fa6b6195ce3264fe275d71f071" exitCode=0 Jul 06 14:30:06.597324 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:06.596862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" event={"ID":"428ced71-a72c-44e1-9c5e-3f1171e83df4","Type":"ContainerDied","Data":"93a9f0c35a827a232eeac74dda0d2fc608e0e7fa6b6195ce3264fe275d71f071"} Jul 06 14:30:07.740092 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:07.740069 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:07.840246 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:07.840220 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsmgc\" (UniqueName: \"kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc\") pod \"428ced71-a72c-44e1-9c5e-3f1171e83df4\" (UID: \"428ced71-a72c-44e1-9c5e-3f1171e83df4\") " Jul 06 14:30:07.842410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:07.842386 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc" (OuterVolumeSpecName: "kube-api-access-fsmgc") pod "428ced71-a72c-44e1-9c5e-3f1171e83df4" (UID: "428ced71-a72c-44e1-9c5e-3f1171e83df4"). InnerVolumeSpecName "kube-api-access-fsmgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:30:07.941497 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:07.941403 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsmgc\" (UniqueName: \"kubernetes.io/projected/428ced71-a72c-44e1-9c5e-3f1171e83df4-kube-api-access-fsmgc\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:30:08.613513 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:08.613480 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" Jul 06 14:30:08.613717 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:08.613517 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722470-dh2sw" event={"ID":"428ced71-a72c-44e1-9c5e-3f1171e83df4","Type":"ContainerDied","Data":"6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f"} Jul 06 14:30:08.613717 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:30:08.613574 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6399eb4e026e55b0c783c161951a8e9f1caf731d856683261298af5c6ed2bc8f" Jul 06 14:34:31.135819 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:31.135773 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:34:31.136451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:31.136066 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" podUID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" containerName="manager" containerID="cri-o://76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6" gracePeriod=10 Jul 06 14:34:34.798146 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:34.798121 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:34:34.922220 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:34.922108 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gplcm\" (UniqueName: \"kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm\") pod \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " Jul 06 14:34:34.922220 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:34.922188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume\") pod \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\" (UID: \"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1\") " Jul 06 14:34:34.922589 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:34.922569 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" (UID: "4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:34:34.924330 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:34.924298 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm" (OuterVolumeSpecName: "kube-api-access-gplcm") pod "4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" (UID: "4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1"). InnerVolumeSpecName "kube-api-access-gplcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:34:35.023661 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.023622 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-extensions-socket-volume\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:34:35.023661 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.023657 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gplcm\" (UniqueName: \"kubernetes.io/projected/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1-kube-api-access-gplcm\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:34:35.141564 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.141533 2581 generic.go:356] "Generic (PLEG): container finished" podID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" containerID="76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6" exitCode=0 Jul 06 14:34:35.141803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.141581 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" Jul 06 14:34:35.141803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.141626 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" event={"ID":"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1","Type":"ContainerDied","Data":"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6"} Jul 06 14:34:35.141803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.141678 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn" event={"ID":"4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1","Type":"ContainerDied","Data":"6703b4d536339e8f0af908d280d21b948f31ab395dfc0cd0e9663ca92fc3f3e3"} Jul 06 14:34:35.141803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.141700 2581 scope.go:117] "RemoveContainer" containerID="76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6" Jul 06 14:34:35.152340 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.152323 2581 scope.go:117] "RemoveContainer" containerID="76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6" Jul 06 14:34:35.152581 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:34:35.152563 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6\": container with ID starting with 76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6 not found: ID does not exist" containerID="76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6" Jul 06 14:34:35.152624 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.152590 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6"} err="failed to get container status \"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6\": rpc error: code = NotFound desc = could not find container \"76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6\": container with ID starting with 76f06590f8e486e71a91d8a580190630ab50a3646f7ac181c15577fbd5384da6 not found: ID does not exist" Jul 06 14:34:35.167221 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.167180 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:34:35.174782 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:35.174724 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-jh5wn"] Jul 06 14:34:36.813902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:36.813863 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" path="/var/lib/kubelet/pods/4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1/volumes" Jul 06 14:34:44.769874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:44.769842 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:34:44.772732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:34:44.772708 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:35:32.255878 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.255841 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws"] Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256470 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" containerName="manager" Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256491 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" containerName="manager" Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256513 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428ced71-a72c-44e1-9c5e-3f1171e83df4" containerName="cleanup" Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256521 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="428ced71-a72c-44e1-9c5e-3f1171e83df4" containerName="cleanup" Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256641 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d57a80b-9fb5-4c4d-8f2e-ab27e05653f1" containerName="manager" Jul 06 14:35:32.258559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.256656 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="428ced71-a72c-44e1-9c5e-3f1171e83df4" containerName="cleanup" Jul 06 14:35:32.260033 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.260015 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.264519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.264501 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6vfj7\"" Jul 06 14:35:32.278620 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.278594 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws"] Jul 06 14:35:32.298398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.298369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/392152eb-142d-4446-883e-f16659f03506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.298398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.298401 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htl4g\" (UniqueName: \"kubernetes.io/projected/392152eb-142d-4446-883e-f16659f03506-kube-api-access-htl4g\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.399492 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.399459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/392152eb-142d-4446-883e-f16659f03506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.399492 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.399491 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htl4g\" (UniqueName: \"kubernetes.io/projected/392152eb-142d-4446-883e-f16659f03506-kube-api-access-htl4g\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.399906 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.399886 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/392152eb-142d-4446-883e-f16659f03506-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.412727 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.412698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htl4g\" (UniqueName: \"kubernetes.io/projected/392152eb-142d-4446-883e-f16659f03506-kube-api-access-htl4g\") pod \"kuadrant-operator-controller-manager-86878c769f-2b9ws\" (UID: \"392152eb-142d-4446-883e-f16659f03506\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.569958 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.569870 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:32.726816 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:32.726791 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws"] Jul 06 14:35:32.729280 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:35:32.729252 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392152eb_142d_4446_883e_f16659f03506.slice/crio-3bb52be217196c15d328b54faec0fd5fb286bf7d4b7f317449317acbd38ceeeb WatchSource:0}: Error finding container 3bb52be217196c15d328b54faec0fd5fb286bf7d4b7f317449317acbd38ceeeb: Status 404 returned error can't find the container with id 3bb52be217196c15d328b54faec0fd5fb286bf7d4b7f317449317acbd38ceeeb Jul 06 14:35:33.465850 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:33.465811 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" event={"ID":"392152eb-142d-4446-883e-f16659f03506","Type":"ContainerStarted","Data":"f0144a30ea7d8d4e084fb0e5d5ae30c12e15d381a928ec446fe6bceb1bf84ef9"} Jul 06 14:35:33.466266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:33.465870 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" event={"ID":"392152eb-142d-4446-883e-f16659f03506","Type":"ContainerStarted","Data":"3bb52be217196c15d328b54faec0fd5fb286bf7d4b7f317449317acbd38ceeeb"} Jul 06 14:35:33.466266 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:33.465900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:35:33.493366 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:33.493320 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" podStartSLOduration=1.4933063739999999 podStartE2EDuration="1.493306374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:35:33.489916169 +0000 UTC m=+1249.184756422" watchObservedRunningTime="2026-07-06 14:35:33.493306374 +0000 UTC m=+1249.188146620" Jul 06 14:35:44.474320 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:35:44.474235 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2b9ws" Jul 06 14:39:44.829058 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:39:44.829017 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:39:44.830176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:39:44.830159 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:41:35.421111 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.421078 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:35.425184 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.425164 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.434059 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.434043 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6f5d260e-openshift-default-dockercfg-7vc8w\"" Jul 06 14:41:35.474785 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.474744 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:35.482729 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.482876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.482876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.482876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482791 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.482876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482815 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.483105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482892 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.483105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482943 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.483105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.483105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.482988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7hpj\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.583981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.583894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.583981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.583933 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.583981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.583955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.583981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.583980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7hpj\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584085 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584334 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584441 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584692 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584609 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.584745 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.584704 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.585450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.585419 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.586604 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.586581 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.586760 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.586746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.593814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.593794 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.594241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.594214 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7hpj\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj\") pod \"e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.736252 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.736217 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:35.887427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.887402 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:35.888778 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:41:35.888754 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0801550a_74a4_4b31_87d3_e92d3c51201f.slice/crio-42b41c28ab7b7b0d01f90517b9cad00a62b16bdd1b2dab658a506bd3db286651 WatchSource:0}: Error finding container 42b41c28ab7b7b0d01f90517b9cad00a62b16bdd1b2dab658a506bd3db286651: Status 404 returned error can't find the container with id 42b41c28ab7b7b0d01f90517b9cad00a62b16bdd1b2dab658a506bd3db286651 Jul 06 14:41:35.890824 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.890795 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:35.890901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.890858 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:35.890901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:35.890886 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:36.565139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:36.565097 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" event={"ID":"0801550a-74a4-4b31-87d3-e92d3c51201f","Type":"ContainerStarted","Data":"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98"} Jul 06 14:41:36.565633 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:36.565152 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" event={"ID":"0801550a-74a4-4b31-87d3-e92d3c51201f","Type":"ContainerStarted","Data":"42b41c28ab7b7b0d01f90517b9cad00a62b16bdd1b2dab658a506bd3db286651"} Jul 06 14:41:36.589564 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:36.589509 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" podStartSLOduration=1.589489135 podStartE2EDuration="1.589489135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:41:36.587956652 +0000 UTC m=+1612.282796932" watchObservedRunningTime="2026-07-06 14:41:36.589489135 +0000 UTC m=+1612.284329384" Jul 06 14:41:36.736981 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:36.736936 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:36.742750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:36.742722 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:37.570439 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:37.570410 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:37.571523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:37.571504 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:41.906165 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:41.906123 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:41.906696 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:41.906361 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" podUID="0801550a-74a4-4b31-87d3-e92d3c51201f" containerName="istio-proxy" containerID="cri-o://8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98" gracePeriod=30 Jul 06 14:41:47.166079 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.166013 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:47.296818 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296785 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296860 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296881 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296945 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.296980 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297300 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297023 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297300 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297051 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297300 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297076 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7hpj\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj\") pod \"0801550a-74a4-4b31-87d3-e92d3c51201f\" (UID: \"0801550a-74a4-4b31-87d3-e92d3c51201f\") " Jul 06 14:41:47.297300 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297126 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:41:47.297502 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297334 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:41:47.297502 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297388 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.297502 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297390 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:41:47.297502 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297488 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:41:47.297733 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.297489 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:41:47.299236 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.299175 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:41:47.299236 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.299174 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:41:47.299441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.299419 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:41:47.299535 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.299520 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj" (OuterVolumeSpecName: "kube-api-access-w7hpj") pod "0801550a-74a4-4b31-87d3-e92d3c51201f" (UID: "0801550a-74a4-4b31-87d3-e92d3c51201f"). InnerVolumeSpecName "kube-api-access-w7hpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:41:47.398488 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398459 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398488 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398484 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398488 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398493 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0801550a-74a4-4b31-87d3-e92d3c51201f-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398501 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398510 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7hpj\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-kube-api-access-w7hpj\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398518 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398526 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0801550a-74a4-4b31-87d3-e92d3c51201f-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.398697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.398533 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0801550a-74a4-4b31-87d3-e92d3c51201f-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:41:47.627509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.627472 2581 generic.go:356] "Generic (PLEG): container finished" podID="0801550a-74a4-4b31-87d3-e92d3c51201f" containerID="8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98" exitCode=0 Jul 06 14:41:47.627705 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.627526 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" Jul 06 14:41:47.627705 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.627559 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" event={"ID":"0801550a-74a4-4b31-87d3-e92d3c51201f","Type":"ContainerDied","Data":"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98"} Jul 06 14:41:47.627705 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.627625 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n" event={"ID":"0801550a-74a4-4b31-87d3-e92d3c51201f","Type":"ContainerDied","Data":"42b41c28ab7b7b0d01f90517b9cad00a62b16bdd1b2dab658a506bd3db286651"} Jul 06 14:41:47.627705 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.627646 2581 scope.go:117] "RemoveContainer" containerID="8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98" Jul 06 14:41:47.637754 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.637738 2581 scope.go:117] "RemoveContainer" containerID="8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98" Jul 06 14:41:47.637978 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:41:47.637957 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98\": container with ID starting with 8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98 not found: ID does not exist" containerID="8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98" Jul 06 14:41:47.638042 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.637990 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98"} err="failed to get container status \"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98\": rpc error: code = NotFound desc = could not find container \"8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98\": container with ID starting with 8ea1d2246769f2efee017e1c5e9a966f1f1d29425a7f116e248ebe378f703e98 not found: ID does not exist" Jul 06 14:41:47.653756 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.653731 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:47.656476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:47.656455 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6f5d260e-openshift-default-5b88886875-b4z8n"] Jul 06 14:41:48.814721 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:48.814683 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0801550a-74a4-4b31-87d3-e92d3c51201f" path="/var/lib/kubelet/pods/0801550a-74a4-4b31-87d3-e92d3c51201f/volumes" Jul 06 14:41:49.174592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.174511 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:41:49.175012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.174993 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0801550a-74a4-4b31-87d3-e92d3c51201f" containerName="istio-proxy" Jul 06 14:41:49.175106 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.175015 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0801550a-74a4-4b31-87d3-e92d3c51201f" containerName="istio-proxy" Jul 06 14:41:49.175163 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.175121 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0801550a-74a4-4b31-87d3-e92d3c51201f" containerName="istio-proxy" Jul 06 14:41:49.179944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.179923 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.184877 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.184854 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-15e000e5-openshift-default-dockercfg-8t9kq\"" Jul 06 14:41:49.193584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.193558 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:41:49.312549 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312514 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312549 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312549 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312603 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312621 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312636 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312701 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkgx\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.312803 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312780 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.313028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.312806 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413524 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkgx\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413569 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413593 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413709 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413697 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413756 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413803 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.413988 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413971 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.414222 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.413973 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.414222 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.414180 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.414415 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.414397 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.414514 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.414496 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.416686 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.416663 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.416889 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.416872 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.424628 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.424570 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkgx\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.424790 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.424771 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token\") pod \"e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.490106 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.490077 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:49.653608 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.653577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:41:49.653977 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:41:49.653934 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5646898_328c_4b38_b422_587def071ade.slice/crio-32bef3188d3ab34ec23e58223d055a58138a73a81be387ca188a3e7d9bb35f98 WatchSource:0}: Error finding container 32bef3188d3ab34ec23e58223d055a58138a73a81be387ca188a3e7d9bb35f98: Status 404 returned error can't find the container with id 32bef3188d3ab34ec23e58223d055a58138a73a81be387ca188a3e7d9bb35f98 Jul 06 14:41:49.656269 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.656231 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:49.656344 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.656320 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:49.656388 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:49.656371 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:41:50.647537 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:50.647493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerStarted","Data":"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956"} Jul 06 14:41:50.647949 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:50.647567 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerStarted","Data":"32bef3188d3ab34ec23e58223d055a58138a73a81be387ca188a3e7d9bb35f98"} Jul 06 14:41:50.681961 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:50.681910 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" podStartSLOduration=1.6818892399999998 podStartE2EDuration="1.68188924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:41:50.681648969 +0000 UTC m=+1626.376489216" watchObservedRunningTime="2026-07-06 14:41:50.68188924 +0000 UTC m=+1626.376729484" Jul 06 14:41:51.490375 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:51.490340 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:51.495989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:51.495962 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:51.653031 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:51.652997 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:41:51.654063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:41:51.654043 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:42:01.718446 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.718409 2581 generic.go:356] "Generic (PLEG): container finished" podID="f5646898-328c-4b38-b422-587def071ade" containerID="30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956" exitCode=0 Jul 06 14:42:01.718881 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.718483 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerDied","Data":"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956"} Jul 06 14:42:01.718881 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.718862 2581 scope.go:117] "RemoveContainer" containerID="30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956" Jul 06 14:42:01.719876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.719849 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:01.719951 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.719919 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:01.719991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.719970 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:01.988681 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:01.988587 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:42:02.725492 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:02.725447 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerStarted","Data":"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27"} Jul 06 14:42:02.725927 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:02.725498 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" containerID="cri-o://48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27" gracePeriod=30 Jul 06 14:42:06.653269 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:06.653186 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:42:07.982638 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:07.982584 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:42:08.089485 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089450 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089536 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089571 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089618 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089688 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089722 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089773 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpkgx\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.089862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089796 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket\") pod \"f5646898-328c-4b38-b422-587def071ade\" (UID: \"f5646898-328c-4b38-b422-587def071ade\") " Jul 06 14:42:08.090115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.089898 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:08.090252 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.090121 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.090252 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.090160 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:08.090393 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.090240 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:42:08.090393 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.090315 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:08.090956 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.090880 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:08.092530 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.092474 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:42:08.092636 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.092526 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:42:08.092636 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.092582 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:08.092914 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.092889 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx" (OuterVolumeSpecName: "kube-api-access-vpkgx") pod "f5646898-328c-4b38-b422-587def071ade" (UID: "f5646898-328c-4b38-b422-587def071ade"). InnerVolumeSpecName "kube-api-access-vpkgx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:42:08.191420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191386 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191413 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191421 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5646898-328c-4b38-b422-587def071ade-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191429 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5646898-328c-4b38-b422-587def071ade-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191439 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191446 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpkgx\" (UniqueName: \"kubernetes.io/projected/f5646898-328c-4b38-b422-587def071ade-kube-api-access-vpkgx\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191456 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.191679 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.191465 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5646898-328c-4b38-b422-587def071ade-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:08.761366 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.761329 2581 generic.go:356] "Generic (PLEG): container finished" podID="f5646898-328c-4b38-b422-587def071ade" containerID="48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27" exitCode=0 Jul 06 14:42:08.761536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.761390 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" Jul 06 14:42:08.761536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.761421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerDied","Data":"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27"} Jul 06 14:42:08.761536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.761476 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf" event={"ID":"f5646898-328c-4b38-b422-587def071ade","Type":"ContainerDied","Data":"32bef3188d3ab34ec23e58223d055a58138a73a81be387ca188a3e7d9bb35f98"} Jul 06 14:42:08.761536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.761503 2581 scope.go:117] "RemoveContainer" containerID="48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27" Jul 06 14:42:08.771420 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.771402 2581 scope.go:117] "RemoveContainer" containerID="30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956" Jul 06 14:42:08.779774 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.779748 2581 scope.go:117] "RemoveContainer" containerID="48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27" Jul 06 14:42:08.780013 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:42:08.779994 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27\": container with ID starting with 48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27 not found: ID does not exist" containerID="48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27" Jul 06 14:42:08.780057 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.780023 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27"} err="failed to get container status \"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27\": rpc error: code = NotFound desc = could not find container \"48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27\": container with ID starting with 48b25a7ec846c47ef865ed43068c0f3cac7c4ee97f2bac3c9c19dabdf24fdd27 not found: ID does not exist" Jul 06 14:42:08.780057 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.780040 2581 scope.go:117] "RemoveContainer" containerID="30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956" Jul 06 14:42:08.780321 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:42:08.780303 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956\": container with ID starting with 30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956 not found: ID does not exist" containerID="30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956" Jul 06 14:42:08.780366 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.780330 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956"} err="failed to get container status \"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956\": rpc error: code = NotFound desc = could not find container \"30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956\": container with ID starting with 30a1be6415f28a9310b5000af005ae130b0815dab4128dd33ea8cd48802ea956 not found: ID does not exist" Jul 06 14:42:08.806332 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.806305 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:42:08.815611 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:08.815585 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-15e000e5-openshift-default-65cf8d997-sxbxf"] Jul 06 14:42:09.327166 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327129 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:09.327769 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327748 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.327814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327774 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.327814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327799 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.327814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327809 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.327932 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327919 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.327968 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.327939 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5646898-328c-4b38-b422-587def071ade" containerName="istio-proxy" Jul 06 14:42:09.337904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.332348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.338608 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.338581 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5e4b6a62-openshift-default-dockercfg-nfrmj\"" Jul 06 14:42:09.351698 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.351672 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:09.400206 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400159 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slnmk\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400331 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400416 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.400542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.400442 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501585 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slnmk\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.501757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501745 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.502103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.501978 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.502103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.502000 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.502646 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.502489 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.502646 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.502553 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.502818 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.502650 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.504153 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.504131 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.504472 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.504446 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.516647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.516620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.518566 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.518548 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slnmk\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk\") pod \"e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.649387 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.649297 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:09.793072 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.793045 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:09.794158 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:42:09.794134 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6578c6dc_9fc5_4448_b1d8_1b249c12eb3c.slice/crio-886b2e72147e83795b3f2d40aa3d7c53b1258c763cb50b1e3890c0077df53706 WatchSource:0}: Error finding container 886b2e72147e83795b3f2d40aa3d7c53b1258c763cb50b1e3890c0077df53706: Status 404 returned error can't find the container with id 886b2e72147e83795b3f2d40aa3d7c53b1258c763cb50b1e3890c0077df53706 Jul 06 14:42:09.796611 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.796566 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:09.796721 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.796653 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:09.796721 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:09.796698 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:42:10.776997 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:10.776961 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" event={"ID":"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c","Type":"ContainerStarted","Data":"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1"} Jul 06 14:42:10.777411 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:10.777013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" event={"ID":"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c","Type":"ContainerStarted","Data":"886b2e72147e83795b3f2d40aa3d7c53b1258c763cb50b1e3890c0077df53706"} Jul 06 14:42:10.813368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:10.813309 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" podStartSLOduration=1.813287886 podStartE2EDuration="1.813287886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:42:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:42:10.807433909 +0000 UTC m=+1646.502274156" watchObservedRunningTime="2026-07-06 14:42:10.813287886 +0000 UTC m=+1646.508128135" Jul 06 14:42:10.817553 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:10.817525 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5646898-328c-4b38-b422-587def071ade" path="/var/lib/kubelet/pods/f5646898-328c-4b38-b422-587def071ade/volumes" Jul 06 14:42:11.650425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:11.650391 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:11.655131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:11.655103 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:11.782536 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:11.782506 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:11.783562 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:11.783545 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:15.256305 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:15.256272 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:15.256763 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:15.256445 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" podUID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" containerName="istio-proxy" containerID="cri-o://abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1" gracePeriod=30 Jul 06 14:42:20.512516 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.512490 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:20.599495 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599411 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599495 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599473 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599499 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599531 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599553 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599718 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599698 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599727 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slnmk\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.599926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599742 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:20.599926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599866 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:20.599926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data\") pod \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\" (UID: \"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c\") " Jul 06 14:42:20.600224 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599975 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:20.600224 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.599990 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:42:20.600522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.600236 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data" (OuterVolumeSpecName: "istio-data") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:20.600522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.600273 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.600522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.600286 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.600522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.600293 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.600522 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.600300 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.601958 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.601929 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:42:20.602063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.602006 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token" (OuterVolumeSpecName: "istio-token") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:42:20.602120 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.602060 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk" (OuterVolumeSpecName: "kube-api-access-slnmk") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "kube-api-access-slnmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:42:20.602172 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.602130 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" (UID: "6578c6dc-9fc5-4448-b1d8-1b249c12eb3c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:42:20.700835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.700807 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.700835 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.700832 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.701003 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.700840 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.701003 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.700850 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slnmk\" (UniqueName: \"kubernetes.io/projected/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-kube-api-access-slnmk\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.701003 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.700860 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:42:20.839374 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.839341 2581 generic.go:356] "Generic (PLEG): container finished" podID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" containerID="abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1" exitCode=0 Jul 06 14:42:20.839503 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.839386 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" Jul 06 14:42:20.839503 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.839425 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" event={"ID":"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c","Type":"ContainerDied","Data":"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1"} Jul 06 14:42:20.839503 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.839479 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw" event={"ID":"6578c6dc-9fc5-4448-b1d8-1b249c12eb3c","Type":"ContainerDied","Data":"886b2e72147e83795b3f2d40aa3d7c53b1258c763cb50b1e3890c0077df53706"} Jul 06 14:42:20.839503 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.839502 2581 scope.go:117] "RemoveContainer" containerID="abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1" Jul 06 14:42:20.851495 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.851470 2581 scope.go:117] "RemoveContainer" containerID="abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1" Jul 06 14:42:20.851968 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:42:20.851942 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1\": container with ID starting with abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1 not found: ID does not exist" containerID="abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1" Jul 06 14:42:20.852073 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.851977 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1"} err="failed to get container status \"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1\": rpc error: code = NotFound desc = could not find container \"abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1\": container with ID starting with abf49af1ef064bcae352897d55c05492635f3afe2bae92b25be20f6cb3d2c3f1 not found: ID does not exist" Jul 06 14:42:20.860241 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.860218 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:20.865314 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:20.865294 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5e4b6a62-openshift-default-78dfbd78c-7xmcw"] Jul 06 14:42:22.816432 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:42:22.816397 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" path="/var/lib/kubelet/pods/6578c6dc-9fc5-4448-b1d8-1b249c12eb3c/volumes" Jul 06 14:44:10.037290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.037260 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:10.037761 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.037639 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" containerName="istio-proxy" Jul 06 14:44:10.037761 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.037650 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" containerName="istio-proxy" Jul 06 14:44:10.037761 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.037714 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6578c6dc-9fc5-4448-b1d8-1b249c12eb3c" containerName="istio-proxy" Jul 06 14:44:10.039848 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.039832 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.043500 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.043477 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-45508226-openshift-default-dockercfg-b9txq\"" Jul 06 14:44:10.086098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086100 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086123 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086254 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086283 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086307 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086359 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086392 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmz4s\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.086458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.086444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.097805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.097772 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:10.187556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187581 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187688 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.187805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.187708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmz4s\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.188235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.188040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.188235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.188166 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.188235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.188211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.188235 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.188229 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.188395 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.188341 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.190159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.190140 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.190398 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.190381 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.198571 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.198525 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.198865 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.198844 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmz4s\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s\") pod \"e2e-mt-45508226-openshift-default-564475d847-cck49\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.350254 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.350137 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:10.510519 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.510487 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:10.513527 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:44:10.513487 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd88ecb_9d6e_4dcb_9e98_90f83dc03fa8.slice/crio-2ff76a888d89f09aeb0e91124c14c393b3ed6c5ea6486f6244b9aae2e7217060 WatchSource:0}: Error finding container 2ff76a888d89f09aeb0e91124c14c393b3ed6c5ea6486f6244b9aae2e7217060: Status 404 returned error can't find the container with id 2ff76a888d89f09aeb0e91124c14c393b3ed6c5ea6486f6244b9aae2e7217060 Jul 06 14:44:10.515558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.515516 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:10.515635 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.515610 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:10.515677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:10.515643 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:11.460694 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:11.460654 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerStarted","Data":"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc"} Jul 06 14:44:11.461167 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:11.460721 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerStarted","Data":"2ff76a888d89f09aeb0e91124c14c393b3ed6c5ea6486f6244b9aae2e7217060"} Jul 06 14:44:11.504695 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:11.504639 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podStartSLOduration=1.5046250620000001 podStartE2EDuration="1.504625062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:44:11.502946992 +0000 UTC m=+1767.197787240" watchObservedRunningTime="2026-07-06 14:44:11.504625062 +0000 UTC m=+1767.199465309" Jul 06 14:44:12.350382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:12.350349 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:12.355092 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:12.355067 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:12.473904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:12.473871 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:12.474942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:12.474919 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:26.566131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.566045 2581 generic.go:356] "Generic (PLEG): container finished" podID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerID="17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc" exitCode=0 Jul 06 14:44:26.566543 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.566121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerDied","Data":"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc"} Jul 06 14:44:26.566591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.566544 2581 scope.go:117] "RemoveContainer" containerID="17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc" Jul 06 14:44:26.567628 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.567596 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:26.567701 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.567684 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:26.567750 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:26.567734 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:27.474034 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:27.474000 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:27.575556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:27.575520 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerStarted","Data":"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047"} Jul 06 14:44:28.350976 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:28.350946 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:28.351902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:28.351878 2581 patch_prober.go:28] interesting pod/e2e-mt-45508226-openshift-default-564475d847-cck49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 14:44:28.352024 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:28.351926 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 14:44:29.351314 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:29.351284 2581 patch_prober.go:28] interesting pod/e2e-mt-45508226-openshift-default-564475d847-cck49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 14:44:29.351688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:29.351342 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 14:44:30.351106 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:30.351070 2581 patch_prober.go:28] interesting pod/e2e-mt-45508226-openshift-default-564475d847-cck49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 14:44:30.351302 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:30.351126 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 14:44:31.350956 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:31.350919 2581 patch_prober.go:28] interesting pod/e2e-mt-45508226-openshift-default-564475d847-cck49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 14:44:31.351353 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:31.350980 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 14:44:32.354846 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:32.354814 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:32.355327 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:32.355175 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:32.355897 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:32.355877 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:34.818280 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:34.818250 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:34.818747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:34.818415 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" containerID="cri-o://e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047" gracePeriod=30 Jul 06 14:44:40.076073 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.076048 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:40.168907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168824 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.168907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168877 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmz4s\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.168907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168897 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168935 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168954 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.168976 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169004 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169040 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169162 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169106 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token\") pod \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\" (UID: \"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8\") " Jul 06 14:44:40.169491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169328 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:44:40.169491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169338 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:44:40.169491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169437 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:44:40.169654 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.169489 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:44:40.170056 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.170014 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:44:40.171451 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.171428 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:44:40.171543 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.171447 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:44:40.171642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.171618 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:44:40.171642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.171627 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s" (OuterVolumeSpecName: "kube-api-access-kmz4s") pod "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" (UID: "9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8"). InnerVolumeSpecName "kube-api-access-kmz4s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:44:40.270731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270701 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270726 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmz4s\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-kube-api-access-kmz4s\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270737 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270746 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270754 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270762 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270769 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270776 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.270947 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.270783 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:44:40.651457 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.651424 2581 generic.go:356] "Generic (PLEG): container finished" podID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerID="e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047" exitCode=0 Jul 06 14:44:40.651631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.651471 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" Jul 06 14:44:40.651631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.651516 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerDied","Data":"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047"} Jul 06 14:44:40.651631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.651567 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49" event={"ID":"9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8","Type":"ContainerDied","Data":"2ff76a888d89f09aeb0e91124c14c393b3ed6c5ea6486f6244b9aae2e7217060"} Jul 06 14:44:40.651631 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.651590 2581 scope.go:117] "RemoveContainer" containerID="e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047" Jul 06 14:44:40.661403 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.661386 2581 scope.go:117] "RemoveContainer" containerID="17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc" Jul 06 14:44:40.669704 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.669680 2581 scope.go:117] "RemoveContainer" containerID="e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047" Jul 06 14:44:40.669951 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:44:40.669933 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047\": container with ID starting with e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047 not found: ID does not exist" containerID="e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047" Jul 06 14:44:40.670028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.669967 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047"} err="failed to get container status \"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047\": rpc error: code = NotFound desc = could not find container \"e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047\": container with ID starting with e75cd90feb5e2aa517e28b82187d6325ad867e7d71e6b2e26ed1310eefdf6047 not found: ID does not exist" Jul 06 14:44:40.670028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.669992 2581 scope.go:117] "RemoveContainer" containerID="17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc" Jul 06 14:44:40.670281 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:44:40.670262 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc\": container with ID starting with 17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc not found: ID does not exist" containerID="17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc" Jul 06 14:44:40.670329 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.670288 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc"} err="failed to get container status \"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc\": rpc error: code = NotFound desc = could not find container \"17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc\": container with ID starting with 17b4d7bf893a4c6d235683ff2d25d1178d1b6add2c50a4f0a37cd2924ea0b0fc not found: ID does not exist" Jul 06 14:44:40.679354 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.679331 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:40.690781 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.690756 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-45508226-openshift-default-564475d847-cck49"] Jul 06 14:44:40.815983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:40.815954 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" path="/var/lib/kubelet/pods/9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8/volumes" Jul 06 14:44:44.868348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:44.868320 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:44:44.871977 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:44.871942 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:44:55.991124 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991091 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:44:55.991518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991497 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.991518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991509 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.991595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991524 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.991595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991530 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.991655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991602 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.991655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.991609 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd88ecb-9d6e-4dcb-9e98-90f83dc03fa8" containerName="istio-proxy" Jul 06 14:44:55.993588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.993573 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:55.996946 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:55.996919 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-94615ad1-openshift-default-dockercfg-t2vx6\"" Jul 06 14:44:56.010052 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.010025 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:44:56.119408 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119595 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119539 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119637 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119689 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119721 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmjxp\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.119807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.119751 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.220684 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmjxp\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.220851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.220851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.220851 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220829 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220868 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220895 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220956 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221019 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.220988 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221300 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.221225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221404 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.221379 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221538 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.221397 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221538 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.221479 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.221643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.221616 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.223305 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.223283 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.223611 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.223594 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.233048 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.232982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmjxp\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.234341 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.234320 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token\") pod \"e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.306167 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.306089 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:56.465987 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:44:56.465939 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e591d64_4979_41d3_8392_2da4c9d3006c.slice/crio-7896158b7c6981fcbb0b2f5ba290d671e8b8e32583c47084d26cd5bd4cd482ba WatchSource:0}: Error finding container 7896158b7c6981fcbb0b2f5ba290d671e8b8e32583c47084d26cd5bd4cd482ba: Status 404 returned error can't find the container with id 7896158b7c6981fcbb0b2f5ba290d671e8b8e32583c47084d26cd5bd4cd482ba Jul 06 14:44:56.466892 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.466867 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:44:56.468050 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.468020 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:56.468139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.468079 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:56.468139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.468112 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:44:56.751772 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.751724 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerStarted","Data":"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a"} Jul 06 14:44:56.751983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.751800 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerStarted","Data":"7896158b7c6981fcbb0b2f5ba290d671e8b8e32583c47084d26cd5bd4cd482ba"} Jul 06 14:44:56.781396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:56.781329 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podStartSLOduration=1.781306472 podStartE2EDuration="1.781306472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:44:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:44:56.776933667 +0000 UTC m=+1812.471773915" watchObservedRunningTime="2026-07-06 14:44:56.781306472 +0000 UTC m=+1812.476146722" Jul 06 14:44:57.307210 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:57.307168 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:57.312102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:57.312079 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:57.757580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:57.757548 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:44:57.758610 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:44:57.758591 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:45:10.859961 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.859927 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerID="0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a" exitCode=0 Jul 06 14:45:10.860471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.860010 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerDied","Data":"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a"} Jul 06 14:45:10.860471 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.860448 2581 scope.go:117] "RemoveContainer" containerID="0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a" Jul 06 14:45:10.861466 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.861426 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:45:10.861589 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.861501 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:45:10.861589 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:10.861532 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:45:11.867067 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:11.867033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerStarted","Data":"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f"} Jul 06 14:45:12.306584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:12.306542 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:45:12.307692 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:12.307654 2581 patch_prober.go:28] interesting pod/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 14:45:12.307820 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:12.307724 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 14:45:12.757704 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:12.757672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:45:13.306882 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:13.306838 2581 patch_prober.go:28] interesting pod/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 14:45:13.307386 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:13.306911 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 14:45:14.306853 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:14.306804 2581 patch_prober.go:28] interesting pod/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 14:45:14.307040 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:14.306888 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 14:45:15.307320 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:15.307278 2581 patch_prober.go:28] interesting pod/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 14:45:15.307810 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:15.307360 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 14:45:16.310167 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:16.310130 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:45:16.311301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:45:16.311287 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:46:35.066588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:35.066547 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:46:35.067066 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:35.066746 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" containerID="cri-o://6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f" gracePeriod=30 Jul 06 14:46:40.333288 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.333213 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:46:40.354944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.354912 2581 generic.go:356] "Generic (PLEG): container finished" podID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerID="6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f" exitCode=0 Jul 06 14:46:40.355110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.354974 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" Jul 06 14:46:40.355110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.354998 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerDied","Data":"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f"} Jul 06 14:46:40.355110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.355080 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr" event={"ID":"3e591d64-4979-41d3-8392-2da4c9d3006c","Type":"ContainerDied","Data":"7896158b7c6981fcbb0b2f5ba290d671e8b8e32583c47084d26cd5bd4cd482ba"} Jul 06 14:46:40.355248 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.355113 2581 scope.go:117] "RemoveContainer" containerID="6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f" Jul 06 14:46:40.366067 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.366045 2581 scope.go:117] "RemoveContainer" containerID="0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a" Jul 06 14:46:40.377708 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.377681 2581 scope.go:117] "RemoveContainer" containerID="6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f" Jul 06 14:46:40.378054 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:46:40.378025 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f\": container with ID starting with 6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f not found: ID does not exist" containerID="6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f" Jul 06 14:46:40.378164 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.378069 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f"} err="failed to get container status \"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f\": rpc error: code = NotFound desc = could not find container \"6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f\": container with ID starting with 6a01f10971c1a1f5b735272fdbd6b6c9d3fc9628c3ea7806f1a1499b47fa107f not found: ID does not exist" Jul 06 14:46:40.378164 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.378098 2581 scope.go:117] "RemoveContainer" containerID="0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a" Jul 06 14:46:40.378437 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:46:40.378412 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a\": container with ID starting with 0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a not found: ID does not exist" containerID="0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a" Jul 06 14:46:40.378510 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.378445 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a"} err="failed to get container status \"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a\": rpc error: code = NotFound desc = could not find container \"0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a\": container with ID starting with 0b448d6cf664e435c8307002fc1f28d250214a494e9e5f2525a8212dd781976a not found: ID does not exist" Jul 06 14:46:40.457098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457065 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457116 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457145 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457185 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457233 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457292 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457269 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmjxp\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457336 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457465 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457392 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457533 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457468 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket\") pod \"3e591d64-4979-41d3-8392-2da4c9d3006c\" (UID: \"3e591d64-4979-41d3-8392-2da4c9d3006c\") " Jul 06 14:46:40.457961 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.457935 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:46:40.459360 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.458358 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:46:40.459360 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.458434 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:46:40.459360 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.458552 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:46:40.459360 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.459268 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:46:40.460998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.460962 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:46:40.461290 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.461265 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp" (OuterVolumeSpecName: "kube-api-access-rmjxp") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "kube-api-access-rmjxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:46:40.461627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.461588 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:46:40.461864 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.461846 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e591d64-4979-41d3-8392-2da4c9d3006c" (UID: "3e591d64-4979-41d3-8392-2da4c9d3006c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:46:40.558346 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558314 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558346 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558344 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558346 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558354 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558362 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558370 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e591d64-4979-41d3-8392-2da4c9d3006c-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558378 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmjxp\" (UniqueName: \"kubernetes.io/projected/3e591d64-4979-41d3-8392-2da4c9d3006c-kube-api-access-rmjxp\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558386 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558396 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.558575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.558405 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e591d64-4979-41d3-8392-2da4c9d3006c-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:46:40.699036 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.698999 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:46:40.718920 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.718883 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-94615ad1-openshift-default-5bcd84d6ff-794hr"] Jul 06 14:46:40.816939 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:46:40.816906 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" path="/var/lib/kubelet/pods/3e591d64-4979-41d3-8392-2da4c9d3006c/volumes" Jul 06 14:47:34.894441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.894400 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:47:34.894901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.894831 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:34.894901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.894847 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:34.894986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.894952 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:34.894986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.894963 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:34.897128 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.897110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:34.902973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.902944 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-17d239-openshift-default-dockercfg-kcl56\"" Jul 06 14:47:34.921307 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:34.921269 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:47:35.028671 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.028671 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028670 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.028907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8svfk\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.028907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.028907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.028907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.029094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028945 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.029094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.028976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.029094 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.029074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130188 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8svfk\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130374 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130645 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130450 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.130645 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130472 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.131026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.130999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.131415 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.131385 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.131527 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.131401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.138702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.131490 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.138702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.131623 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.138702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.132158 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.138702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.134558 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.138702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.135129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.148715 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.148665 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.156044 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.156021 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8svfk\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk\") pod \"e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.208148 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.208113 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:35.364491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.364460 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:47:35.365458 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:47:35.365426 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c405c2_2c2d_4e3c_ab9b_7b40352411f9.slice/crio-1800b6bafb9f51426a3abbde31fb4c595aa2f570a1f74be0602dd1ad83cbc7c0 WatchSource:0}: Error finding container 1800b6bafb9f51426a3abbde31fb4c595aa2f570a1f74be0602dd1ad83cbc7c0: Status 404 returned error can't find the container with id 1800b6bafb9f51426a3abbde31fb4c595aa2f570a1f74be0602dd1ad83cbc7c0 Jul 06 14:47:35.367489 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.367456 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:35.367591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.367535 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:35.367591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.367579 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:35.660578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.660455 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerStarted","Data":"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51"} Jul 06 14:47:35.660578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.660507 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerStarted","Data":"1800b6bafb9f51426a3abbde31fb4c595aa2f570a1f74be0602dd1ad83cbc7c0"} Jul 06 14:47:35.692855 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:35.692798 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podStartSLOduration=1.6927775330000001 podStartE2EDuration="1.692777533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:47:35.689296879 +0000 UTC m=+1971.384137138" watchObservedRunningTime="2026-07-06 14:47:35.692777533 +0000 UTC m=+1971.387617783" Jul 06 14:47:36.208737 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:36.208706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:36.210011 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:36.209987 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 14:47:36.210127 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:36.210037 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 14:47:37.213514 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:37.213482 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:37.672242 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:37.672141 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:37.673257 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:37.673238 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:43.277708 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.277671 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:47:43.278092 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.278070 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:43.278092 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.278082 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e591d64-4979-41d3-8392-2da4c9d3006c" containerName="istio-proxy" Jul 06 14:47:43.280659 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.280642 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.288458 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.288434 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2bc2ad-openshift-default-dockercfg-f8h47\"" Jul 06 14:47:43.316224 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.314447 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:47:43.414457 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.414414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.414776 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.414755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvs5x\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.414916 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.414899 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415087 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415072 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415276 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415262 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415428 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415549 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415533 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415656 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.415791 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.415776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516733 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516765 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516895 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvs5x\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.516999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.516970 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.517309 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.517024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.517309 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.517059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.517882 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.517854 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.518804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.518297 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.518950 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.518533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.519061 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.518746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.519157 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.518918 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.522441 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.522415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.524442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.524414 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.528525 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.528466 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.531639 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.531615 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvs5x\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x\") pod \"e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.591523 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.591493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:43.813872 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.811253 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:47:43.827403 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.827362 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:43.827526 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.827446 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:43.827526 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:43.827487 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:44.750029 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:44.749986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerStarted","Data":"aa4a4718855c03a8a09061aa9fb6bfb3b4ff8d4f76ca878539f608e960d45b42"} Jul 06 14:47:44.750494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:44.750081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerStarted","Data":"8b425c987f3ba5d74c1aca05355eb570946db4b71a8f2f90e0a32642ac625004"} Jul 06 14:47:44.889913 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:44.889856 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podStartSLOduration=1.889836221 podStartE2EDuration="1.889836221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:47:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:47:44.882952892 +0000 UTC m=+1980.577793140" watchObservedRunningTime="2026-07-06 14:47:44.889836221 +0000 UTC m=+1980.584676469" Jul 06 14:47:45.591859 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:45.591826 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:45.608312 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:45.608270 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:45.755685 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:45.755652 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:45.756686 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:45.756668 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:47:50.786267 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.786224 2581 generic.go:356] "Generic (PLEG): container finished" podID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerID="780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51" exitCode=0 Jul 06 14:47:50.786693 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.786297 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerDied","Data":"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51"} Jul 06 14:47:50.786738 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.786706 2581 scope.go:117] "RemoveContainer" containerID="780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51" Jul 06 14:47:50.787708 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.787682 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:50.787772 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.787759 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:50.787820 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:50.787809 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:51.807146 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:51.806107 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerStarted","Data":"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92"} Jul 06 14:47:52.208299 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:52.208248 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:52.209666 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:52.209633 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 14:47:52.209917 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:52.209873 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 14:47:52.672698 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:52.672596 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:53.209629 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:53.209588 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 14:47:53.210283 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:53.210249 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 14:47:54.209033 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:54.208993 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 14:47:54.209368 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:54.209326 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 14:47:55.209210 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:55.209140 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 14:47:55.209754 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:55.209234 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 14:47:56.221701 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:56.221665 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:56.224892 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:56.224861 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:47:59.876986 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.876910 2581 generic.go:356] "Generic (PLEG): container finished" podID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerID="aa4a4718855c03a8a09061aa9fb6bfb3b4ff8d4f76ca878539f608e960d45b42" exitCode=0 Jul 06 14:47:59.877389 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.876983 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerDied","Data":"aa4a4718855c03a8a09061aa9fb6bfb3b4ff8d4f76ca878539f608e960d45b42"} Jul 06 14:47:59.877453 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.877436 2581 scope.go:117] "RemoveContainer" containerID="aa4a4718855c03a8a09061aa9fb6bfb3b4ff8d4f76ca878539f608e960d45b42" Jul 06 14:47:59.878501 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.878475 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:59.878575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.878557 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:47:59.878614 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:47:59.878599 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:48:00.756541 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:00.756504 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:48:00.884825 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:00.884791 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerStarted","Data":"e9af2604847f23ace0935b09eb5f30744590c72dd28b3ab9088cfa85dee11996"} Jul 06 14:48:01.592636 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:01.592593 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:48:01.593812 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:01.593785 2581 patch_prober.go:28] interesting pod/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 06 14:48:01.593945 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:01.593838 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 06 14:48:02.592665 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:02.592626 2581 patch_prober.go:28] interesting pod/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 06 14:48:02.593088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:02.592685 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 06 14:48:03.591883 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:03.591846 2581 patch_prober.go:28] interesting pod/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 06 14:48:03.592085 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:03.591899 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 06 14:48:04.592351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:04.592312 2581 patch_prober.go:28] interesting pod/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 06 14:48:04.592766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:04.592376 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 06 14:48:05.596145 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:05.596116 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:48:05.596611 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:05.596590 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:48:05.597233 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:48:05.597215 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:49:41.210943 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:41.210900 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:49:41.211497 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:41.211169 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" containerID="cri-o://4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92" gracePeriod=30 Jul 06 14:49:41.226338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:41.226304 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:49:41.226475 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:41.226372 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:49:44.906103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:44.906067 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:49:44.911949 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:44.911926 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:49:46.469547 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.469478 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:49:46.510436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.510397 2581 generic.go:356] "Generic (PLEG): container finished" podID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerID="4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92" exitCode=0 Jul 06 14:49:46.510619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.510447 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" Jul 06 14:49:46.510619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.510474 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerDied","Data":"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92"} Jul 06 14:49:46.510619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.510532 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t" event={"ID":"52c405c2-2c2d-4e3c-ab9b-7b40352411f9","Type":"ContainerDied","Data":"1800b6bafb9f51426a3abbde31fb4c595aa2f570a1f74be0602dd1ad83cbc7c0"} Jul 06 14:49:46.510619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.510565 2581 scope.go:117] "RemoveContainer" containerID="4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92" Jul 06 14:49:46.523288 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.523229 2581 scope.go:117] "RemoveContainer" containerID="780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51" Jul 06 14:49:46.532623 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.532600 2581 scope.go:117] "RemoveContainer" containerID="4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92" Jul 06 14:49:46.532908 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:49:46.532884 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92\": container with ID starting with 4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92 not found: ID does not exist" containerID="4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92" Jul 06 14:49:46.533006 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.532916 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92"} err="failed to get container status \"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92\": rpc error: code = NotFound desc = could not find container \"4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92\": container with ID starting with 4af54747639455e8a1b3f6c0aa067dbfa8915384a5c73e1b5c330a43df335b92 not found: ID does not exist" Jul 06 14:49:46.533006 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.532936 2581 scope.go:117] "RemoveContainer" containerID="780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51" Jul 06 14:49:46.533243 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:49:46.533221 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51\": container with ID starting with 780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51 not found: ID does not exist" containerID="780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51" Jul 06 14:49:46.533315 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.533252 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51"} err="failed to get container status \"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51\": rpc error: code = NotFound desc = could not find container \"780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51\": container with ID starting with 780ac1518ef58349521523830d31883d7feed56a6571ef5e5ab608c8636d9a51 not found: ID does not exist" Jul 06 14:49:46.571885 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.571858 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.571902 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.571938 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.571979 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.571997 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572262 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572026 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572262 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572107 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572262 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572152 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8svfk\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572262 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572174 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token\") pod \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\" (UID: \"52c405c2-2c2d-4e3c-ab9b-7b40352411f9\") " Jul 06 14:49:46.572476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572299 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:46.572476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572285 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:46.572476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572319 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:46.572476 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572350 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:46.572658 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572495 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.572658 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572511 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.572658 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.572526 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.573247 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.573221 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:46.574309 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.574281 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:49:46.574478 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.574455 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:46.574920 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.574894 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:46.574920 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.574907 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk" (OuterVolumeSpecName: "kube-api-access-8svfk") pod "52c405c2-2c2d-4e3c-ab9b-7b40352411f9" (UID: "52c405c2-2c2d-4e3c-ab9b-7b40352411f9"). InnerVolumeSpecName "kube-api-access-8svfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:46.673801 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673765 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.673801 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673793 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8svfk\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-kube-api-access-8svfk\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.673801 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673803 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.673801 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673811 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.674062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673819 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.674062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.673827 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52c405c2-2c2d-4e3c-ab9b-7b40352411f9-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:46.844573 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.844541 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:49:46.853131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:46.853104 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-17d239-openshift-default-9665b847c-4pl8t"] Jul 06 14:49:48.814926 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:48.814884 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" path="/var/lib/kubelet/pods/52c405c2-2c2d-4e3c-ab9b-7b40352411f9/volumes" Jul 06 14:49:50.328494 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:50.328465 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:49:50.328915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:50.328682 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" containerID="cri-o://e9af2604847f23ace0935b09eb5f30744590c72dd28b3ab9088cfa85dee11996" gracePeriod=30 Jul 06 14:49:50.598601 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:50.598509 2581 patch_prober.go:28] interesting pod/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:49:50.598601 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:50.598565 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:49:51.347571 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.347529 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348137 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348158 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348174 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348182 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348348 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.348584 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.348368 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="52c405c2-2c2d-4e3c-ab9b-7b40352411f9" containerName="istio-proxy" Jul 06 14:49:51.350801 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.350779 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.355350 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.355329 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a89965-openshift-default-dockercfg-qvjdj\"" Jul 06 14:49:51.370707 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.370683 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:49:51.419188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419152 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419188 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419187 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419302 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419338 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419425 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slxj8\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419461 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419495 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.419521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.419512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520486 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520486 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520510 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520592 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520621 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520663 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520692 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.520732 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slxj8\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.521103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520911 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.521103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.520986 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.521103 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.521055 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.521379 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.521360 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.521442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.521426 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.523706 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.523679 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.523869 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.523848 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.536055 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.536027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slxj8\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.541222 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.541181 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token\") pod \"e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.661364 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.661266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:51.820031 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.820007 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:49:51.821061 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:49:51.821037 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc2df9e3_ce6f_4b09_99e1_4578afb4468a.slice/crio-2cd539c6f3013bd29229e696dc6e70a951fefca6f3223581f865d2e89bf66195 WatchSource:0}: Error finding container 2cd539c6f3013bd29229e696dc6e70a951fefca6f3223581f865d2e89bf66195: Status 404 returned error can't find the container with id 2cd539c6f3013bd29229e696dc6e70a951fefca6f3223581f865d2e89bf66195 Jul 06 14:49:51.823123 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.823096 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:49:51.823211 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.823160 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:49:51.823256 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:51.823210 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:49:52.546908 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:52.546867 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerStarted","Data":"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db"} Jul 06 14:49:52.547285 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:52.546931 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerStarted","Data":"2cd539c6f3013bd29229e696dc6e70a951fefca6f3223581f865d2e89bf66195"} Jul 06 14:49:52.585790 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:52.585738 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podStartSLOduration=1.585722334 podStartE2EDuration="1.585722334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:49:52.582876704 +0000 UTC m=+2108.277716953" watchObservedRunningTime="2026-07-06 14:49:52.585722334 +0000 UTC m=+2108.280562580" Jul 06 14:49:52.662260 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:52.662221 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:52.667291 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:52.667259 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:53.552900 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:53.552870 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:53.553869 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:53.553849 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:49:55.567176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.567098 2581 generic.go:356] "Generic (PLEG): container finished" podID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerID="e9af2604847f23ace0935b09eb5f30744590c72dd28b3ab9088cfa85dee11996" exitCode=0 Jul 06 14:49:55.567176 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.567170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerDied","Data":"e9af2604847f23ace0935b09eb5f30744590c72dd28b3ab9088cfa85dee11996"} Jul 06 14:49:55.567841 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.567260 2581 scope.go:117] "RemoveContainer" containerID="aa4a4718855c03a8a09061aa9fb6bfb3b4ff8d4f76ca878539f608e960d45b42" Jul 06 14:49:55.602252 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.602169 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:49:55.761397 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761368 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761593 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761434 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761593 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761461 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761593 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761477 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761774 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761594 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761774 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761640 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761774 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761717 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761774 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761751 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvs5x\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761768 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:55.761989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761795 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:55.761989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761849 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy\") pod \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\" (UID: \"dd20e91a-1924-4a5d-82c1-1a6cb59a8883\") " Jul 06 14:49:55.761989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761933 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:55.761989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.761942 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:49:55.762455 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.762170 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.762455 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.762219 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.762455 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.762242 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.762455 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.762255 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.762727 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.762700 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data" (OuterVolumeSpecName: "istio-data") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:55.763989 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.763944 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:49:55.764108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.764053 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x" (OuterVolumeSpecName: "kube-api-access-hvs5x") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "kube-api-access-hvs5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:55.764108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.764024 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:49:55.764108 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.764041 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token" (OuterVolumeSpecName: "istio-token") pod "dd20e91a-1924-4a5d-82c1-1a6cb59a8883" (UID: "dd20e91a-1924-4a5d-82c1-1a6cb59a8883"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:49:55.863382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.863301 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.863382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.863327 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvs5x\" (UniqueName: \"kubernetes.io/projected/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-kube-api-access-hvs5x\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.863382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.863336 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.863382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.863347 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:55.863382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:55.863359 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd20e91a-1924-4a5d-82c1-1a6cb59a8883-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:49:56.574346 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.574319 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" Jul 06 14:49:56.574730 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.574362 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg" event={"ID":"dd20e91a-1924-4a5d-82c1-1a6cb59a8883","Type":"ContainerDied","Data":"8b425c987f3ba5d74c1aca05355eb570946db4b71a8f2f90e0a32642ac625004"} Jul 06 14:49:56.574730 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.574420 2581 scope.go:117] "RemoveContainer" containerID="e9af2604847f23ace0935b09eb5f30744590c72dd28b3ab9088cfa85dee11996" Jul 06 14:49:56.624427 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.624391 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:49:56.631098 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.631063 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2bc2ad-openshift-default-5c6f4d48b7-pq4tg"] Jul 06 14:49:56.815688 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:49:56.815654 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" path="/var/lib/kubelet/pods/dd20e91a-1924-4a5d-82c1-1a6cb59a8883/volumes" Jul 06 14:50:00.452355 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452322 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:00.452757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452714 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:00.452757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452725 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:00.452757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452735 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:00.452757 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452742 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:00.452928 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.452809 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:00.456253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.456235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.465696 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.465673 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f2544a-openshift-default-dockercfg-gpvnr\"" Jul 06 14:50:00.526941 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.526880 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:00.607915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.607867 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608225 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608402 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608385 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608537 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608675 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608860 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.608991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.608976 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74rm\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.609230 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.609212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.609383 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.609368 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710622 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710622 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710580 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710865 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710865 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710865 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710719 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.710865 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.711099 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rm\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.711099 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.710943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.711099 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.711005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.711682 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.711655 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.713234 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.711893 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.713459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.712071 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.713580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.712246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.713680 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.712792 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.723491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.723455 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.723862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.723760 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.728373 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.728346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74rm\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.730862 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.730834 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token\") pod \"e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.766590 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.766559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:00.991129 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:00.991049 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:01.009550 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:50:01.008337 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f677c4d_790f_4554_869c_18f6fdc4efda.slice/crio-32091c706340ff994b04a91999ca322a1fb2e285b4a4e736c0b442588231e230 WatchSource:0}: Error finding container 32091c706340ff994b04a91999ca322a1fb2e285b4a4e736c0b442588231e230: Status 404 returned error can't find the container with id 32091c706340ff994b04a91999ca322a1fb2e285b4a4e736c0b442588231e230 Jul 06 14:50:01.010468 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.010410 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:01.010580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.010512 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:01.010580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.010557 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:01.615770 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.615723 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerStarted","Data":"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b"} Jul 06 14:50:01.616317 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.615798 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerStarted","Data":"32091c706340ff994b04a91999ca322a1fb2e285b4a4e736c0b442588231e230"} Jul 06 14:50:01.672759 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.672702 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podStartSLOduration=1.6726866729999998 podStartE2EDuration="1.672686673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:50:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:50:01.670766974 +0000 UTC m=+2117.365607222" watchObservedRunningTime="2026-07-06 14:50:01.672686673 +0000 UTC m=+2117.367526920" Jul 06 14:50:01.766852 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:01.766814 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:02.772139 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:02.772111 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:03.631510 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:03.631474 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:03.632436 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:03.632414 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:07.657904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.657867 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerID="2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db" exitCode=0 Jul 06 14:50:07.658424 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.657942 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerDied","Data":"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db"} Jul 06 14:50:07.658424 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.658391 2581 scope.go:117] "RemoveContainer" containerID="2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db" Jul 06 14:50:07.659421 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.659393 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:07.659517 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.659459 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:07.659517 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:07.659490 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:08.553730 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:08.553605 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:08.670129 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:08.670095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerStarted","Data":"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26"} Jul 06 14:50:09.661651 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:09.661612 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:09.662588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:09.662553 2581 patch_prober.go:28] interesting pod/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 06 14:50:09.662704 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:09.662620 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 06 14:50:10.662691 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:10.662648 2581 patch_prober.go:28] interesting pod/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 06 14:50:10.663110 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:10.662723 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 06 14:50:11.661942 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:11.661902 2581 patch_prober.go:28] interesting pod/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Jul 06 14:50:11.662131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:11.661963 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Jul 06 14:50:12.666168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:12.666137 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:12.666575 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:12.666510 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:12.667164 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:12.667148 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:16.718912 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.718877 2581 generic.go:356] "Generic (PLEG): container finished" podID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerID="536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b" exitCode=0 Jul 06 14:50:16.719380 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.718952 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerDied","Data":"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b"} Jul 06 14:50:16.719474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.719456 2581 scope.go:117] "RemoveContainer" containerID="536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b" Jul 06 14:50:16.720499 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.720471 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:16.720603 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.720552 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:16.720667 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:16.720602 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:17.730075 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:17.730042 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerStarted","Data":"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d"} Jul 06 14:50:17.767667 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:17.767633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:17.769126 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:17.769093 2581 patch_prober.go:28] interesting pod/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 06 14:50:17.769351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:17.769176 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 06 14:50:18.631825 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:18.631785 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:18.767706 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:18.767669 2581 patch_prober.go:28] interesting pod/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 06 14:50:18.768088 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:18.767726 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 06 14:50:19.767687 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:19.767655 2581 patch_prober.go:28] interesting pod/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 06 14:50:19.767867 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:19.767707 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 06 14:50:20.767082 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:20.767047 2581 patch_prober.go:28] interesting pod/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Jul 06 14:50:20.767318 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:20.767104 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Jul 06 14:50:21.817344 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:21.817314 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:21.818758 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:21.818699 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:24.366123 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:24.366083 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:50:24.366766 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:24.366333 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" containerID="cri-o://b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26" gracePeriod=30 Jul 06 14:50:27.668863 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:27.668816 2581 patch_prober.go:28] interesting pod/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:50:27.669294 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:27.668886 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:50:29.626878 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.626820 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:29.719563 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719522 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719563 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719565 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719593 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719618 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719659 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slxj8\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719749 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.719827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719812 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.720159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719838 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.720159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719862 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:29.720159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719892 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo\") pod \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\" (UID: \"dc2df9e3-ce6f-4b09-99e1-4578afb4468a\") " Jul 06 14:50:29.720159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.719935 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:29.720159 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720081 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:29.720438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720327 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.720438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720350 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.720438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720366 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.720586 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720493 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:50:29.720706 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.720683 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:29.722038 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.722012 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:50:29.722353 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.722333 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8" (OuterVolumeSpecName: "kube-api-access-slxj8") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "kube-api-access-slxj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:50:29.722559 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.722538 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:29.722649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.722562 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc2df9e3-ce6f-4b09-99e1-4578afb4468a" (UID: "dc2df9e3-ce6f-4b09-99e1-4578afb4468a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:50:29.800979 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.800948 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerID="b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26" exitCode=0 Jul 06 14:50:29.801154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.801017 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" Jul 06 14:50:29.801154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.801036 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerDied","Data":"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26"} Jul 06 14:50:29.801154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.801090 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r" event={"ID":"dc2df9e3-ce6f-4b09-99e1-4578afb4468a","Type":"ContainerDied","Data":"2cd539c6f3013bd29229e696dc6e70a951fefca6f3223581f865d2e89bf66195"} Jul 06 14:50:29.801154 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.801112 2581 scope.go:117] "RemoveContainer" containerID="b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26" Jul 06 14:50:29.811722 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.811704 2581 scope.go:117] "RemoveContainer" containerID="2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db" Jul 06 14:50:29.820049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820031 2581 scope.go:117] "RemoveContainer" containerID="b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26" Jul 06 14:50:29.820340 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:50:29.820321 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26\": container with ID starting with b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26 not found: ID does not exist" containerID="b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26" Jul 06 14:50:29.820392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820349 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26"} err="failed to get container status \"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26\": rpc error: code = NotFound desc = could not find container \"b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26\": container with ID starting with b74a9cdaf0dea679da97d5a94659c8059d5b4856ed274034f0fc96181e075c26 not found: ID does not exist" Jul 06 14:50:29.820392 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820365 2581 scope.go:117] "RemoveContainer" containerID="2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db" Jul 06 14:50:29.820593 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:50:29.820576 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db\": container with ID starting with 2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db not found: ID does not exist" containerID="2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db" Jul 06 14:50:29.820630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820600 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db"} err="failed to get container status \"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db\": rpc error: code = NotFound desc = could not find container \"2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db\": container with ID starting with 2480b2c65a4bedaa5b6f1f9be98d7bcc0bf43e8af4bb3f7d3c1dc243af3413db not found: ID does not exist" Jul 06 14:50:29.820804 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820792 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.820840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820809 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.820840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820818 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slxj8\" (UniqueName: \"kubernetes.io/projected/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-kube-api-access-slxj8\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.820840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820826 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.820840 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820834 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.820966 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.820841 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc2df9e3-ce6f-4b09-99e1-4578afb4468a-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:29.833973 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.833951 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:50:29.837891 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:29.837871 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a89965-openshift-default-6fbfb55c47-ljx4r"] Jul 06 14:50:30.814443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:30.814410 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" path="/var/lib/kubelet/pods/dc2df9e3-ce6f-4b09-99e1-4578afb4468a/volumes" Jul 06 14:50:32.470543 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:32.470511 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:32.470969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:32.470732 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" containerID="cri-o://81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d" gracePeriod=30 Jul 06 14:50:33.353325 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.353290 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:50:33.353780 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.353763 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:33.353827 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.353783 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:33.353876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.353866 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:33.353910 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.353880 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd20e91a-1924-4a5d-82c1-1a6cb59a8883" containerName="istio-proxy" Jul 06 14:50:33.357560 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.357543 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.361972 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.361953 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e9ff6b-openshift-default-dockercfg-25hbx\"" Jul 06 14:50:33.386410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.386383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:50:33.454791 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.454753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k88br\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.454858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.454882 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.454961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.455009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.455040 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.455095 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455661 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.455162 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.455661 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.455247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556081 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88br\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556098 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556228 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556270 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.556576 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556412 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.557026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.556674 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.557026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.557004 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.557135 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.557040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.557135 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.557067 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.557246 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.557133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.559028 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.559004 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.559274 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.559258 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.576842 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.576811 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.576954 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.576935 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k88br\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br\") pod \"e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.668099 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.668024 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:33.823283 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.823260 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:50:33.824748 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:50:33.824719 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78cb4b82_9828_4acc_a1f4_5e9f542bd31b.slice/crio-ead80879284b9e23790fbbf7813706de50914edeed60e75fe9c819c55c09f715 WatchSource:0}: Error finding container ead80879284b9e23790fbbf7813706de50914edeed60e75fe9c819c55c09f715: Status 404 returned error can't find the container with id ead80879284b9e23790fbbf7813706de50914edeed60e75fe9c819c55c09f715 Jul 06 14:50:33.826843 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.826814 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:33.826907 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.826874 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:33.826943 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:33.826904 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:34.832211 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:34.832167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerStarted","Data":"d8b41e861088d1be01c707715ffd441f6f3d7e2b76ba211d1dac4ceb9032e2b0"} Jul 06 14:50:34.832593 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:34.832242 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerStarted","Data":"ead80879284b9e23790fbbf7813706de50914edeed60e75fe9c819c55c09f715"} Jul 06 14:50:34.863250 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:34.863183 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podStartSLOduration=1.863167549 podStartE2EDuration="1.863167549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:50:34.860927823 +0000 UTC m=+2150.555768095" watchObservedRunningTime="2026-07-06 14:50:34.863167549 +0000 UTC m=+2150.558007795" Jul 06 14:50:35.669220 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:35.669113 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:35.674121 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:35.674097 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:35.838737 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:35.838704 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:35.839830 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:35.839811 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:36.820438 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:36.820400 2581 patch_prober.go:28] interesting pod/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:50:36.820630 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:36.820457 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:50:37.735944 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.735889 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:37.806996 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.806965 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.806996 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807007 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w74rm\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807051 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807081 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807098 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807132 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807535 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807350 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:37.807535 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807420 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:37.807535 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807499 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:37.807697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807659 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:50:37.807697 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807678 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807726 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.807807 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo\") pod \"8f677c4d-790f-4554-869c-18f6fdc4efda\" (UID: \"8f677c4d-790f-4554-869c-18f6fdc4efda\") " Jul 06 14:50:37.808020 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.807993 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:37.808115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.808041 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.808115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.808056 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.808115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.808069 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.808115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.808081 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f677c4d-790f-4554-869c-18f6fdc4efda-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.810049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.809994 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:50:37.810049 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.810032 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:50:37.810182 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.810093 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm" (OuterVolumeSpecName: "kube-api-access-w74rm") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "kube-api-access-w74rm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:50:37.810400 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.810382 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f677c4d-790f-4554-869c-18f6fdc4efda" (UID: "8f677c4d-790f-4554-869c-18f6fdc4efda"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:50:37.851701 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.851668 2581 generic.go:356] "Generic (PLEG): container finished" podID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerID="81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d" exitCode=0 Jul 06 14:50:37.851892 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.851769 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" Jul 06 14:50:37.851892 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.851808 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerDied","Data":"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d"} Jul 06 14:50:37.851998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.851894 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c" event={"ID":"8f677c4d-790f-4554-869c-18f6fdc4efda","Type":"ContainerDied","Data":"32091c706340ff994b04a91999ca322a1fb2e285b4a4e736c0b442588231e230"} Jul 06 14:50:37.851998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.851925 2581 scope.go:117] "RemoveContainer" containerID="81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d" Jul 06 14:50:37.862687 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.862666 2581 scope.go:117] "RemoveContainer" containerID="536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b" Jul 06 14:50:37.871518 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.871496 2581 scope.go:117] "RemoveContainer" containerID="81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d" Jul 06 14:50:37.871765 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:50:37.871746 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d\": container with ID starting with 81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d not found: ID does not exist" containerID="81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d" Jul 06 14:50:37.871814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.871776 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d"} err="failed to get container status \"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d\": rpc error: code = NotFound desc = could not find container \"81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d\": container with ID starting with 81267979a7396d5bca9c5577425054cc89e14f8f32c08708b8928cd9639ff00d not found: ID does not exist" Jul 06 14:50:37.871814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.871793 2581 scope.go:117] "RemoveContainer" containerID="536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b" Jul 06 14:50:37.872010 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:50:37.871994 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b\": container with ID starting with 536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b not found: ID does not exist" containerID="536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b" Jul 06 14:50:37.872051 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.872017 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b"} err="failed to get container status \"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b\": rpc error: code = NotFound desc = could not find container \"536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b\": container with ID starting with 536581f8eaca0db34365126db8fbe88b97544eb66a0dfe36d119348f46c1101b not found: ID does not exist" Jul 06 14:50:37.894592 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.894557 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:37.904961 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.904935 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f2544a-openshift-default-58446d68df-2pz9c"] Jul 06 14:50:37.909122 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.909090 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.909263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.909128 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.909263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.909153 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.909263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.909166 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f677c4d-790f-4554-869c-18f6fdc4efda-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:37.909263 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:37.909179 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w74rm\" (UniqueName: \"kubernetes.io/projected/8f677c4d-790f-4554-869c-18f6fdc4efda-kube-api-access-w74rm\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:50:38.815141 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:38.815110 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" path="/var/lib/kubelet/pods/8f677c4d-790f-4554-869c-18f6fdc4efda/volumes" Jul 06 14:50:42.089550 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.089513 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:50:42.090473 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090444 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.090619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090608 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.090776 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090765 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:42.090857 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090846 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:42.090934 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090924 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.091007 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.090998 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.091411 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.091393 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc2df9e3-ce6f-4b09-99e1-4578afb4468a" containerName="istio-proxy" Jul 06 14:50:42.091556 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.091544 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.091650 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.091640 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f677c4d-790f-4554-869c-18f6fdc4efda" containerName="istio-proxy" Jul 06 14:50:42.100647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.100617 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.105012 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.104987 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-91c777-openshift-default-dockercfg-s42kv\"" Jul 06 14:50:42.123876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.123849 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:50:42.143903 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.143874 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.143918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.143950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xg88\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.143969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.143990 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.144005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144063 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.144023 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144375 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.144060 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.144375 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.144112 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245313 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245278 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245358 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245391 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245419 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xg88\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245480 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245498 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245747 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.245857 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.245805 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.246121 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.246093 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.246284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.246117 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.246284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.246154 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.246423 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.246188 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.248917 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.248881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.249378 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.249356 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.256410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.256386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.257987 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.257962 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xg88\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88\") pod \"e2e-shared-b-91c777-openshift-default-696447c55-pmr6b\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.412315 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.412231 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:42.575856 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.575831 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:50:42.576674 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:50:42.576644 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1875c525_5025_4aaf_9bdd_6bf5f47c19a9.slice/crio-142772385e835fa74a19ece304da8409e92cc05df2721869dba5d94bfb9dd041 WatchSource:0}: Error finding container 142772385e835fa74a19ece304da8409e92cc05df2721869dba5d94bfb9dd041: Status 404 returned error can't find the container with id 142772385e835fa74a19ece304da8409e92cc05df2721869dba5d94bfb9dd041 Jul 06 14:50:42.578869 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.578842 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:42.578932 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.578905 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:42.578969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.578930 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:42.882904 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.882868 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerStarted","Data":"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383"} Jul 06 14:50:42.883105 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.882929 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerStarted","Data":"142772385e835fa74a19ece304da8409e92cc05df2721869dba5d94bfb9dd041"} Jul 06 14:50:42.914386 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:42.914337 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podStartSLOduration=0.914322459 podStartE2EDuration="914.322459ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 14:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:50:42.911775522 +0000 UTC m=+2158.606615771" watchObservedRunningTime="2026-07-06 14:50:42.914322459 +0000 UTC m=+2158.609162706" Jul 06 14:50:43.413713 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:43.413672 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:43.424873 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:43.424831 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:43.888542 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:43.888511 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:43.889588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:43.889569 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:50.839998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.839919 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:50.839998 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.839972 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:50.954270 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.954232 2581 generic.go:356] "Generic (PLEG): container finished" podID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerID="d8b41e861088d1be01c707715ffd441f6f3d7e2b76ba211d1dac4ceb9032e2b0" exitCode=0 Jul 06 14:50:50.954442 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.954320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerDied","Data":"d8b41e861088d1be01c707715ffd441f6f3d7e2b76ba211d1dac4ceb9032e2b0"} Jul 06 14:50:50.954759 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.954745 2581 scope.go:117] "RemoveContainer" containerID="d8b41e861088d1be01c707715ffd441f6f3d7e2b76ba211d1dac4ceb9032e2b0" Jul 06 14:50:50.955836 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.955805 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:50.955943 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.955914 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:50.956004 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:50.955961 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:51.962358 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:51.962321 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerStarted","Data":"0dee1f0f5a5fed00197c9f30a2c34c0bb93e7f02eb8380558ee05ceb6a6fbee6"} Jul 06 14:50:52.669354 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:52.669317 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:52.669558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:52.669477 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:52.669558 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:52.669527 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:53.669184 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:53.669144 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:53.669587 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:53.669225 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:54.668587 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:54.668554 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:54.668752 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:54.668607 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:55.668872 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:55.668827 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:55.669388 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:55.668896 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:56.669168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:56.669132 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Jul 06 14:50:56.669591 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:56.669216 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Jul 06 14:50:57.673034 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:57.672936 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:57.673524 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:57.673337 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:57.674145 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:57.674131 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:50:58.010079 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.010046 2581 generic.go:356] "Generic (PLEG): container finished" podID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerID="6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383" exitCode=0 Jul 06 14:50:58.010318 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.010125 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerDied","Data":"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383"} Jul 06 14:50:58.010702 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.010684 2581 scope.go:117] "RemoveContainer" containerID="6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383" Jul 06 14:50:58.011738 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.011712 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:58.011821 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.011794 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:58.011866 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.011853 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 06 14:50:58.888805 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:58.888774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:59.024618 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:59.024312 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerStarted","Data":"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7"} Jul 06 14:50:59.412915 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:59.412874 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:50:59.414055 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:59.414021 2581 patch_prober.go:28] interesting pod/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 06 14:50:59.414332 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:50:59.414302 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 06 14:51:00.413171 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:00.413129 2581 patch_prober.go:28] interesting pod/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 06 14:51:00.413690 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:00.413229 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 06 14:51:01.413301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:01.413263 2581 patch_prober.go:28] interesting pod/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 06 14:51:01.413890 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:01.413860 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 06 14:51:02.413140 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:02.413090 2581 patch_prober.go:28] interesting pod/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 06 14:51:02.413377 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:02.413166 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 06 14:51:03.417586 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:03.417558 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:51:03.417983 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:03.417961 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:51:03.418682 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:51:03.418663 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:54:44.946564 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:54:44.946530 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:54:44.953916 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:54:44.953888 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:57:39.737339 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:39.737305 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:57:39.737901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:39.737537 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" containerID="cri-o://0dee1f0f5a5fed00197c9f30a2c34c0bb93e7f02eb8380558ee05ceb6a6fbee6" gracePeriod=30 Jul 06 14:57:42.675731 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:42.675688 2581 patch_prober.go:28] interesting pod/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:57:42.676087 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:42.675748 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:57:44.977616 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:44.977595 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:57:45.102062 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.101986 2581 scope.go:117] "RemoveContainer" containerID="d8b41e861088d1be01c707715ffd441f6f3d7e2b76ba211d1dac4ceb9032e2b0" Jul 06 14:57:45.111443 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.111425 2581 scope.go:117] "RemoveContainer" containerID="0dee1f0f5a5fed00197c9f30a2c34c0bb93e7f02eb8380558ee05ceb6a6fbee6" Jul 06 14:57:45.132231 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132211 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132243 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132273 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132301 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132292 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132325 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132364 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132490 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132440 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k88br\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132547 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132488 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132547 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132513 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:45.132657 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132570 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo\") pod \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\" (UID: \"78cb4b82-9828-4acc-a1f4-5e9f542bd31b\") " Jul 06 14:57:45.132715 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132595 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:45.132773 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132709 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:45.132901 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132880 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.133001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132891 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:45.133001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132903 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.133001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132946 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.133001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.132903 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data" (OuterVolumeSpecName: "istio-data") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:45.134578 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.134554 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token" (OuterVolumeSpecName: "istio-token") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:45.134768 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.134632 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:45.135253 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.135234 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:57:45.135325 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.135248 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br" (OuterVolumeSpecName: "kube-api-access-k88br") pod "78cb4b82-9828-4acc-a1f4-5e9f542bd31b" (UID: "78cb4b82-9828-4acc-a1f4-5e9f542bd31b"). InnerVolumeSpecName "kube-api-access-k88br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:45.233642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233609 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.233642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233634 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.233642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233643 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.233642 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233651 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.233911 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233661 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.233911 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.233668 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k88br\" (UniqueName: \"kubernetes.io/projected/78cb4b82-9828-4acc-a1f4-5e9f542bd31b-kube-api-access-k88br\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:45.402627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.402538 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerDied","Data":"0dee1f0f5a5fed00197c9f30a2c34c0bb93e7f02eb8380558ee05ceb6a6fbee6"} Jul 06 14:57:45.402627 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.402618 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" event={"ID":"78cb4b82-9828-4acc-a1f4-5e9f542bd31b","Type":"ContainerDied","Data":"ead80879284b9e23790fbbf7813706de50914edeed60e75fe9c819c55c09f715"} Jul 06 14:57:45.402814 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.402559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx" Jul 06 14:57:45.441696 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.441663 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:57:45.447889 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:45.447869 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e9ff6b-openshift-default-558d5cffcf-r7jkx"] Jul 06 14:57:46.815364 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:46.815333 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" path="/var/lib/kubelet/pods/78cb4b82-9828-4acc-a1f4-5e9f542bd31b/volumes" Jul 06 14:57:47.602783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:47.602744 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:57:47.603029 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:47.602999 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" containerID="cri-o://a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7" gracePeriod=30 Jul 06 14:57:48.345336 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:48.345303 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758bdf5c95-5bw5z_f8aa6748-8552-4167-89e6-1591d35c3c2d/authorino/0.log" Jul 06 14:57:48.420168 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:48.420136 2581 patch_prober.go:28] interesting pod/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 14:57:48.420348 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:48.420212 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 14:57:52.858138 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.858110 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:57:52.900874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900811 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xg88\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.900874 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900848 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900876 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900899 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900965 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.900988 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901080 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901047 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901117 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901150 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data\") pod \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\" (UID: \"1875c525-5025-4aaf-9bdd-6bf5f47c19a9\") " Jul 06 14:57:52.901382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901110 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:52.901382 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901329 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:52.901671 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901463 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:52.901671 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901518 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 14:57:52.901902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901759 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istiod-ca-cert\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:52.901902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901873 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-certs\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:52.901902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901886 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-workload-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:52.901902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.901898 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-credential-socket\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:52.902276 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.902207 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:52.903491 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.903464 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:52.903606 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.903504 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 14:57:52.903809 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.903783 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88" (OuterVolumeSpecName: "kube-api-access-7xg88") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "kube-api-access-7xg88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 14:57:52.903999 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:52.903978 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1875c525-5025-4aaf-9bdd-6bf5f47c19a9" (UID: "1875c525-5025-4aaf-9bdd-6bf5f47c19a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 14:57:53.003185 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.003163 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-token\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:53.003185 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.003184 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-data\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:53.003185 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.003206 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xg88\" (UniqueName: \"kubernetes.io/projected/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-kube-api-access-7xg88\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:53.003396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.003217 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-envoy\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:53.003396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.003226 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1875c525-5025-4aaf-9bdd-6bf5f47c19a9-istio-podinfo\") on node \"ip-10-0-131-65.ec2.internal\" DevicePath \"\"" Jul 06 14:57:53.177613 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.177544 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8svfr_89e10ed2-7069-4390-add0-19d9a7a6026f/manager/0.log" Jul 06 14:57:53.315908 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.315875 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xppf8_dbf0d4e2-eb60-443d-8251-a82e632625dd/manager/0.log" Jul 06 14:57:53.449877 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.449842 2581 generic.go:356] "Generic (PLEG): container finished" podID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerID="a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7" exitCode=0 Jul 06 14:57:53.450037 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.449893 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" Jul 06 14:57:53.450037 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.449936 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerDied","Data":"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7"} Jul 06 14:57:53.450037 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.450002 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b" event={"ID":"1875c525-5025-4aaf-9bdd-6bf5f47c19a9","Type":"ContainerDied","Data":"142772385e835fa74a19ece304da8409e92cc05df2721869dba5d94bfb9dd041"} Jul 06 14:57:53.450037 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.450032 2581 scope.go:117] "RemoveContainer" containerID="a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7" Jul 06 14:57:53.460357 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.460328 2581 scope.go:117] "RemoveContainer" containerID="6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383" Jul 06 14:57:53.468764 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.468747 2581 scope.go:117] "RemoveContainer" containerID="a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7" Jul 06 14:57:53.468999 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:57:53.468979 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7\": container with ID starting with a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7 not found: ID does not exist" containerID="a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7" Jul 06 14:57:53.469050 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.469008 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7"} err="failed to get container status \"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7\": rpc error: code = NotFound desc = could not find container \"a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7\": container with ID starting with a34c865fd3958a661e2a2cb13542332ae5590f75abf134ab792a61376b6923f7 not found: ID does not exist" Jul 06 14:57:53.469050 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.469025 2581 scope.go:117] "RemoveContainer" containerID="6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383" Jul 06 14:57:53.469316 ip-10-0-131-65 kubenswrapper[2581]: E0706 14:57:53.469294 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383\": container with ID starting with 6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383 not found: ID does not exist" containerID="6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383" Jul 06 14:57:53.469375 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.469322 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383"} err="failed to get container status \"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383\": rpc error: code = NotFound desc = could not find container \"6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383\": container with ID starting with 6899955ec1ec38548dd68cfd6edfbb63aee02dbdc99c65f573f68972806b2383 not found: ID does not exist" Jul 06 14:57:53.479677 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.479656 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:57:53.483664 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.483634 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-91c777-openshift-default-696447c55-pmr6b"] Jul 06 14:57:53.583396 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.583367 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722470-dh2sw_428ced71-a72c-44e1-9c5e-3f1171e83df4/cleanup/0.log" Jul 06 14:57:53.845026 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.844956 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-bf594b9b9-jkxz9_509cda6e-f444-4596-8556-3a31a03c9da2/manager/0.log" Jul 06 14:57:53.967649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:53.967628 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9z8hp_59f9a3dc-92b9-4d63-b337-92521815c82a/server/0.log" Jul 06 14:57:54.092245 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:54.092214 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7mh8s_7a2dac3a-f47b-48dc-a22e-8329cea33e9d/manager/0.log" Jul 06 14:57:54.356676 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:54.356651 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84ddbc8dff-86ljb_709e5301-fad5-4186-884d-246d61b4014a/copy-manifests/0.log" Jul 06 14:57:54.364483 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:54.364463 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84ddbc8dff-86ljb_709e5301-fad5-4186-884d-246d61b4014a/manager/0.log" Jul 06 14:57:54.819777 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:54.819737 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" path="/var/lib/kubelet/pods/1875c525-5025-4aaf-9bdd-6bf5f47c19a9/volumes" Jul 06 14:57:55.415057 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.415026 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/util/0.log" Jul 06 14:57:55.428134 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.428108 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/pull/0.log" Jul 06 14:57:55.436969 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.436937 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/extract/0.log" Jul 06 14:57:55.549639 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.549614 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/util/0.log" Jul 06 14:57:55.556450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.556429 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/pull/0.log" Jul 06 14:57:55.563680 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.563659 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/extract/0.log" Jul 06 14:57:55.677450 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.677380 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/util/0.log" Jul 06 14:57:55.684024 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.684005 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/pull/0.log" Jul 06 14:57:55.690381 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.690364 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/extract/0.log" Jul 06 14:57:55.810272 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.810250 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/extract/0.log" Jul 06 14:57:55.816516 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.816492 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/util/0.log" Jul 06 14:57:55.823169 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.823151 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/pull/0.log" Jul 06 14:57:55.951985 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:55.951964 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758bdf5c95-5bw5z_f8aa6748-8552-4167-89e6-1591d35c3c2d/authorino/0.log" Jul 06 14:57:56.090783 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:56.090762 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6jxtx_fd79a6ce-4f90-49ed-8808-64a3f8e82398/manager/0.log" Jul 06 14:57:56.213081 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:56.213017 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qw5t2_a3f47229-0c45-4948-9b09-3e1e48272f56/manager/0.log" Jul 06 14:57:56.463001 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:56.462951 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hqdzz_d9d16544-da2d-4601-b300-0e2f3f11b062/registry-server/0.log" Jul 06 14:57:56.596474 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:56.596389 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2b9ws_392152eb-142d-4446-883e-f16659f03506/manager/0.log" Jul 06 14:57:56.862131 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:56.862061 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nfblv_d5ea7f07-7970-48e7-8662-08385f6ce910/manager/0.log" Jul 06 14:57:57.221876 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:57.221848 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llv2_eb9dfc8d-63f5-4f66-9c7a-21d451079b30/istio-proxy/0.log" Jul 06 14:57:57.592655 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:57.592585 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbc98d46f-zzgm2_af9030c5-d31d-4e00-82e6-c29a0a8bdd73/kube-auth-proxy/0.log" Jul 06 14:57:57.722848 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:57.722781 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/1.log" Jul 06 14:57:57.855023 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:57.854940 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-bp4mw_8332bf2d-7242-4bcf-86ec-ec6ebe412f80/payload-pre-processing/0.log" Jul 06 14:57:57.983174 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:57.983145 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-4tsqt_8141b573-fcf4-4afe-af80-b06de3bc027a/payload-processing/0.log" Jul 06 14:57:58.590252 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:58.590225 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-p8nx7_4e7bba0c-3e23-4a6d-b807-3d80d2082118/main/0.log" Jul 06 14:57:58.717588 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:57:58.717559 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-kvrvc_2b9694f7-17de-4760-99f0-742835736b25/main/0.log" Jul 06 14:58:07.095237 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:07.095137 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mjjwx_daf0384f-5fef-48ab-a5e9-219455c47099/global-pull-secret-syncer/0.log" Jul 06 14:58:07.249643 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:07.249612 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-n8wlh_8e036f9e-256a-4a3c-a7c4-d2054a543008/konnectivity-agent/0.log" Jul 06 14:58:07.467910 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:07.467881 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-65.ec2.internal_514e6a5df363bafe96e75142e83be5d6/haproxy/0.log" Jul 06 14:58:12.198902 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.198871 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/extract/0.log" Jul 06 14:58:12.225284 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.225257 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/util/0.log" Jul 06 14:58:12.248064 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.248039 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dvcwp_f8f63ee1-127f-423e-a7a3-389c24bce1ed/pull/0.log" Jul 06 14:58:12.280939 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.280915 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/extract/0.log" Jul 06 14:58:12.305355 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.305332 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/util/0.log" Jul 06 14:58:12.334172 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.334147 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0klc98_22e83895-238a-4c5a-88f9-c48b9b1a7a08/pull/0.log" Jul 06 14:58:12.368965 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.368942 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/extract/0.log" Jul 06 14:58:12.392182 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.392155 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/util/0.log" Jul 06 14:58:12.416580 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.416561 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73p566l_de4f9d86-a7b6-4382-8934-667c9d06325c/pull/0.log" Jul 06 14:58:12.485302 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.485233 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/extract/0.log" Jul 06 14:58:12.575155 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.575126 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/util/0.log" Jul 06 14:58:12.654619 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.654588 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r624c_969e84d1-1627-4b3e-a0e9-217ad3dbdc53/pull/0.log" Jul 06 14:58:12.976363 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:12.976336 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-758bdf5c95-5bw5z_f8aa6748-8552-4167-89e6-1591d35c3c2d/authorino/0.log" Jul 06 14:58:13.014521 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:13.014493 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-6jxtx_fd79a6ce-4f90-49ed-8808-64a3f8e82398/manager/0.log" Jul 06 14:58:13.041607 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:13.041580 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qw5t2_a3f47229-0c45-4948-9b09-3e1e48272f56/manager/0.log" Jul 06 14:58:13.115867 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:13.115833 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hqdzz_d9d16544-da2d-4601-b300-0e2f3f11b062/registry-server/0.log" Jul 06 14:58:13.229861 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:13.229780 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2b9ws_392152eb-142d-4446-883e-f16659f03506/manager/0.log" Jul 06 14:58:13.334294 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:13.334264 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-nfblv_d5ea7f07-7970-48e7-8662-08385f6ce910/manager/0.log" Jul 06 14:58:15.584437 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:15.584399 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tqlhq_73a7b71e-6e3f-4861-b509-da2c6bfc2448/node-exporter/0.log" Jul 06 14:58:15.619117 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:15.619086 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tqlhq_73a7b71e-6e3f-4861-b509-da2c6bfc2448/kube-rbac-proxy/0.log" Jul 06 14:58:15.656568 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:15.656542 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tqlhq_73a7b71e-6e3f-4861-b509-da2c6bfc2448/init-textfile/0.log" Jul 06 14:58:16.245276 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:16.245243 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59dbdc5959-b72qt_6ee90ea7-9e8b-405e-a4bd-3759573e62eb/telemeter-client/0.log" Jul 06 14:58:16.285743 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:16.285718 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59dbdc5959-b72qt_6ee90ea7-9e8b-405e-a4bd-3759573e62eb/reload/0.log" Jul 06 14:58:16.322743 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:16.322722 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-59dbdc5959-b72qt_6ee90ea7-9e8b-405e-a4bd-3759573e62eb/kube-rbac-proxy/0.log" Jul 06 14:58:17.220102 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220070 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt"] Jul 06 14:58:17.220482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220465 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" Jul 06 14:58:17.220482 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220477 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" Jul 06 14:58:17.220554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220491 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220497 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220509 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220554 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220515 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220678 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220574 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" Jul 06 14:58:17.220678 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220582 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220678 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220587 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1875c525-5025-4aaf-9bdd-6bf5f47c19a9" containerName="istio-proxy" Jul 06 14:58:17.220678 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.220596 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="78cb4b82-9828-4acc-a1f4-5e9f542bd31b" containerName="istio-proxy" Jul 06 14:58:17.223722 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.223704 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.226869 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.226846 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xpkfs\"/\"default-dockercfg-lwn9w\"" Jul 06 14:58:17.226991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.226850 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpkfs\"/\"openshift-service-ca.crt\"" Jul 06 14:58:17.226991 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.226850 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xpkfs\"/\"kube-root-ca.crt\"" Jul 06 14:58:17.233186 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.233165 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt"] Jul 06 14:58:17.316683 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.316630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-proc\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.316875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.316708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-podres\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.316875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.316733 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7cc6\" (UniqueName: \"kubernetes.io/projected/b9de131f-d0a8-433e-9040-5747e49aeeeb-kube-api-access-h7cc6\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.316875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.316768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-sys\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.316875 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.316797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-lib-modules\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418145 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418101 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-lib-modules\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418257 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-proc\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-podres\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7cc6\" (UniqueName: \"kubernetes.io/projected/b9de131f-d0a8-433e-9040-5747e49aeeeb-kube-api-access-h7cc6\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418309 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-lib-modules\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418351 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-sys\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418561 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-proc\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418561 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418428 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-sys\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.418561 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.418450 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b9de131f-d0a8-433e-9040-5747e49aeeeb-podres\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.427371 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.427351 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7cc6\" (UniqueName: \"kubernetes.io/projected/b9de131f-d0a8-433e-9040-5747e49aeeeb-kube-api-access-h7cc6\") pod \"perf-node-gather-daemonset-dkqkt\" (UID: \"b9de131f-d0a8-433e-9040-5747e49aeeeb\") " pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.534353 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.534265 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:17.678257 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:17.678230 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt"] Jul 06 14:58:17.678970 ip-10-0-131-65 kubenswrapper[2581]: W0706 14:58:17.678942 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb9de131f_d0a8_433e_9040_5747e49aeeeb.slice/crio-5079910a4889dfc6c239a41ed28613addb9b4af10fd9eedf326eba89ea2231a1 WatchSource:0}: Error finding container 5079910a4889dfc6c239a41ed28613addb9b4af10fd9eedf326eba89ea2231a1: Status 404 returned error can't find the container with id 5079910a4889dfc6c239a41ed28613addb9b4af10fd9eedf326eba89ea2231a1 Jul 06 14:58:18.225848 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.225822 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/2.log" Jul 06 14:58:18.230227 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.230182 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-bz6fs_e8f25865-ef65-43b0-849d-bf8786f97e50/console-operator/3.log" Jul 06 14:58:18.597459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.597377 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" event={"ID":"b9de131f-d0a8-433e-9040-5747e49aeeeb","Type":"ContainerStarted","Data":"fd7c7d3281b4be9065456e3775a11bb5549e54db66ce2e7c5a8e2e12b3af6770"} Jul 06 14:58:18.597459 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.597427 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" event={"ID":"b9de131f-d0a8-433e-9040-5747e49aeeeb","Type":"ContainerStarted","Data":"5079910a4889dfc6c239a41ed28613addb9b4af10fd9eedf326eba89ea2231a1"} Jul 06 14:58:18.597656 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.597484 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:18.621593 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:18.621544 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" podStartSLOduration=1.621531098 podStartE2EDuration="1.621531098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 14:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 14:58:18.620537544 +0000 UTC m=+2614.315377791" watchObservedRunningTime="2026-07-06 14:58:18.621531098 +0000 UTC m=+2614.316371353" Jul 06 14:58:20.449279 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:20.449248 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hz5n9_9769f741-41f9-406f-8361-4164f03fbd8a/dns/0.log" Jul 06 14:58:20.471248 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:20.471224 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hz5n9_9769f741-41f9-406f-8361-4164f03fbd8a/kube-rbac-proxy/0.log" Jul 06 14:58:20.623617 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:20.623588 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sqwhz_135e0dca-d92d-483c-9e81-93ff18d532f4/dns-node-resolver/0.log" Jul 06 14:58:21.401515 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:21.401486 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sqrlm_4e4b8620-7304-4e70-9f89-434c533ae3f0/node-ca/0.log" Jul 06 14:58:22.446509 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.446481 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llv2_eb9dfc8d-63f5-4f66-9c7a-21d451079b30/istio-proxy/0.log" Jul 06 14:58:22.708116 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.708093 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbc98d46f-zzgm2_af9030c5-d31d-4e00-82e6-c29a0a8bdd73/kube-auth-proxy/0.log" Jul 06 14:58:22.740788 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.740766 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/0.log" Jul 06 14:58:22.770149 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.770125 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/1.log" Jul 06 14:58:22.805338 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.805301 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-bp4mw_8332bf2d-7242-4bcf-86ec-ec6ebe412f80/payload-pre-processing/0.log" Jul 06 14:58:22.854119 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:22.854095 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-4tsqt_8141b573-fcf4-4afe-af80-b06de3bc027a/payload-processing/0.log" Jul 06 14:58:23.628970 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:23.628930 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gtpmk_685b830e-dc3d-4669-9524-ee547345608a/serve-healthcheck-canary/0.log" Jul 06 14:58:24.233035 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:24.233004 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcjlq_24889622-5265-4405-8061-b2cd283c091f/kube-rbac-proxy/0.log" Jul 06 14:58:24.256289 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:24.256262 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcjlq_24889622-5265-4405-8061-b2cd283c091f/exporter/0.log" Jul 06 14:58:24.282444 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:24.282419 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcjlq_24889622-5265-4405-8061-b2cd283c091f/extractor/0.log" Jul 06 14:58:24.613030 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:24.612963 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xpkfs/perf-node-gather-daemonset-dkqkt" Jul 06 14:58:26.805413 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:26.805386 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-8svfr_89e10ed2-7069-4390-add0-19d9a7a6026f/manager/0.log" Jul 06 14:58:26.934699 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:26.934675 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xppf8_dbf0d4e2-eb60-443d-8251-a82e632625dd/manager/0.log" Jul 06 14:58:27.006435 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.006402 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722470-dh2sw_428ced71-a72c-44e1-9c5e-3f1171e83df4/cleanup/0.log" Jul 06 14:58:27.153871 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.153845 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-bf594b9b9-jkxz9_509cda6e-f444-4596-8556-3a31a03c9da2/manager/0.log" Jul 06 14:58:27.176648 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.176622 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9z8hp_59f9a3dc-92b9-4d63-b337-92521815c82a/server/0.log" Jul 06 14:58:27.249647 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.249615 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-7mh8s_7a2dac3a-f47b-48dc-a22e-8329cea33e9d/manager/0.log" Jul 06 14:58:27.414410 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.414341 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84ddbc8dff-86ljb_709e5301-fad5-4186-884d-246d61b4014a/manager/0.log" Jul 06 14:58:27.441567 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.441542 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84ddbc8dff-86ljb_709e5301-fad5-4186-884d-246d61b4014a/copy-manifests/0.log" Jul 06 14:58:27.947582 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:27.947557 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llv2_eb9dfc8d-63f5-4f66-9c7a-21d451079b30/istio-proxy/0.log" Jul 06 14:58:28.173127 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:28.173101 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbc98d46f-zzgm2_af9030c5-d31d-4e00-82e6-c29a0a8bdd73/kube-auth-proxy/0.log" Jul 06 14:58:28.196555 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:28.196519 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/0.log" Jul 06 14:58:28.227649 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:28.227587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/1.log" Jul 06 14:58:28.251115 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:28.251091 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-bp4mw_8332bf2d-7242-4bcf-86ec-ec6ebe412f80/payload-pre-processing/0.log" Jul 06 14:58:28.287834 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:28.287814 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-4tsqt_8141b573-fcf4-4afe-af80-b06de3bc027a/payload-processing/0.log" Jul 06 14:58:29.155148 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.155069 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-s6rm7_fed7ccb7-c0c9-4e77-8a91-125e3ad9aead/openshift-lws-operator/0.log" Jul 06 14:58:29.501794 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.501755 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48llv2_eb9dfc8d-63f5-4f66-9c7a-21d451079b30/istio-proxy/0.log" Jul 06 14:58:29.728091 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.728062 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fbc98d46f-zzgm2_af9030c5-d31d-4e00-82e6-c29a0a8bdd73/kube-auth-proxy/0.log" Jul 06 14:58:29.747468 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.747436 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/0.log" Jul 06 14:58:29.781837 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.781767 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-psc62_37bf8f08-e9ec-4541-ad03-b20618f0a6fd/istio-proxy/1.log" Jul 06 14:58:29.804896 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.804864 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-bp4mw_8332bf2d-7242-4bcf-86ec-ec6ebe412f80/payload-pre-processing/0.log" Jul 06 14:58:29.841178 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:29.841149 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-4tsqt_8141b573-fcf4-4afe-af80-b06de3bc027a/payload-processing/0.log" Jul 06 14:58:34.254659 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:34.254630 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2zl8z_22ed4150-a056-4e6d-be0d-3233db98adac/migrator/0.log" Jul 06 14:58:34.279017 ip-10-0-131-65 kubenswrapper[2581]: I0706 14:58:34.278994 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-2zl8z_22ed4150-a056-4e6d-be0d-3233db98adac/graceful-termination/0.log"