Aug 04 22:20:56.947675 ip-10-0-137-148 systemd[1]: Starting Kubernetes Kubelet... Aug 04 22:20:57.329597 ip-10-0-137-148 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 22:20:57.329597 ip-10-0-137-148 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 22:20:57.329597 ip-10-0-137-148 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 22:20:57.329597 ip-10-0-137-148 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 22:20:57.329597 ip-10-0-137-148 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 22:20:57.330285 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.329650 2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 22:20:57.333130 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333114 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.333130 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333130 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333135 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333139 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333143 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333146 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333148 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333151 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333154 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333157 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333160 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333163 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333165 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333168 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333178 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333180 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333184 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333188 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333190 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333194 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.333195 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333198 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333201 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333204 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333207 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333211 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333214 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333217 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333220 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333222 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333225 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333228 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333230 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333232 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333235 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333237 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333240 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333243 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333245 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333248 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333250 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.333736 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333253 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333255 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333258 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333261 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333264 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333266 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333269 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333271 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333274 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333276 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333278 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333281 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333283 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333286 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333290 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333293 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333296 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333298 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333301 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.334241 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333303 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333306 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333308 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333311 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333314 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333316 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333319 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333321 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333324 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333326 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333329 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333331 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333334 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333336 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333339 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333341 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333344 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333346 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333349 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333352 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.334731 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333355 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333357 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333360 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333363 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333366 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333369 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333371 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333374 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333377 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333379 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333382 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333385 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333387 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333390 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333393 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333395 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333398 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333400 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333402 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.333405 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.335232 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334128 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334135 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334138 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334141 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334143 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334148 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334153 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334156 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334158 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334161 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334163 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334166 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334170 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334173 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334176 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334178 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334181 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334184 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334187 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334190 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.335704 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334193 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334196 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334198 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334201 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334204 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334208 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334211 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334214 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334218 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334220 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334223 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334226 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334229 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334233 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334236 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334240 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334243 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334246 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334249 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.336207 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334252 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334255 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334257 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334260 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334263 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334266 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334268 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334271 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334273 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334276 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334278 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334281 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334284 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334286 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334289 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334292 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334294 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334297 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334299 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334302 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.336685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334305 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334307 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334310 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334312 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334314 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334317 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334319 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334322 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334325 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334328 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334330 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334333 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334335 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334338 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334340 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334343 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334345 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334348 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334351 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334353 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.337203 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334356 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334358 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334361 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334364 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334366 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334369 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334372 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334375 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334377 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334380 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334382 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334385 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334387 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334390 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334393 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334395 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334398 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334400 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334403 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334405 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.337678 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334793 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334798 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334800 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334803 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334806 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334808 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334811 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334813 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334816 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334819 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334821 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334824 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334827 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334830 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334832 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334835 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334838 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334841 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334843 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334846 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.338180 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334848 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334851 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334853 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334856 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334858 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334861 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334863 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334866 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334868 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334871 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334873 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334876 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334879 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334882 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334884 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334886 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334889 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334891 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334894 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334896 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.338672 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334900 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334903 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334907 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334911 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334914 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334917 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334919 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334922 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334926 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334929 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334932 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334934 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334937 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334939 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334942 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334944 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334947 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334950 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334952 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334955 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.339185 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334958 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334961 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334963 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334966 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334968 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334971 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334973 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334976 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334978 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334980 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334983 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334985 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334988 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334990 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334993 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334995 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.334998 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335001 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335003 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335006 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.339695 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335011 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335015 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335018 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335020 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335023 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335026 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335029 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335031 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335048 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335051 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335054 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335057 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335060 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335062 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335065 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335067 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335070 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335072 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335075 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335137 2571 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 22:20:57.340177 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335144 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335149 2571 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335153 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335157 2571 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335160 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335164 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335168 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335171 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335174 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335179 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335182 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335187 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335191 2571 flags.go:64] FLAG: --cgroup-root="" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335194 2571 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335197 2571 flags.go:64] FLAG: --client-ca-file="" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335200 2571 flags.go:64] FLAG: --cloud-provider="external" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335202 2571 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335207 2571 flags.go:64] FLAG: --cluster-domain="" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335209 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335212 2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335215 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335219 2571 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335223 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 22:20:57.340656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335225 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335228 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335232 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335235 2571 flags.go:64] FLAG: --contention-profiling="false" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335238 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335241 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335244 2571 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335247 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335251 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335254 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335257 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335260 2571 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335262 2571 flags.go:64] FLAG: --enable-server="true" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335265 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335269 2571 flags.go:64] FLAG: --event-burst="100" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335272 2571 flags.go:64] FLAG: --event-qps="50" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335275 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335277 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335280 2571 flags.go:64] FLAG: --eviction-hard="" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335285 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335289 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335293 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335296 2571 flags.go:64] FLAG: --eviction-soft="" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335299 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335302 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 22:20:57.341219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335305 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335308 2571 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335311 2571 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335314 2571 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335317 2571 flags.go:64] FLAG: --feature-gates="" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335320 2571 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335323 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335326 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335329 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335332 2571 flags.go:64] FLAG: --healthz-port="10248" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335335 2571 flags.go:64] FLAG: --help="false" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335338 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335341 2571 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335344 2571 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335347 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335350 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335354 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335356 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335359 2571 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335362 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335365 2571 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335368 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335371 2571 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335373 2571 flags.go:64] FLAG: --kube-reserved="" Aug 04 22:20:57.341861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335376 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335379 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335382 2571 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335387 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335392 2571 flags.go:64] FLAG: --lock-file="" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335395 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335398 2571 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335401 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335406 2571 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335408 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335411 2571 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335414 2571 flags.go:64] FLAG: --logging-format="text" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335417 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335420 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335423 2571 flags.go:64] FLAG: --manifest-url="" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335426 2571 flags.go:64] FLAG: --manifest-url-header="" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335430 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335432 2571 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335437 2571 flags.go:64] FLAG: --max-pods="110" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335440 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335443 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335446 2571 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335449 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335452 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335455 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 22:20:57.342472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335457 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335464 2571 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335467 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335470 2571 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335473 2571 flags.go:64] FLAG: --pod-cidr="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335476 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335481 2571 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335484 2571 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335487 2571 flags.go:64] FLAG: --pods-per-core="0" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335490 2571 flags.go:64] FLAG: --port="10250" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335493 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335501 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04a90f180aa6218a7" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335505 2571 flags.go:64] FLAG: --qos-reserved="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335508 2571 flags.go:64] FLAG: --read-only-port="10255" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335511 2571 flags.go:64] FLAG: --register-node="true" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335514 2571 flags.go:64] FLAG: --register-with-taints="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335517 2571 flags.go:64] FLAG: --registry-burst="10" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335520 2571 flags.go:64] FLAG: --registry-qps="5" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335523 2571 flags.go:64] FLAG: --reserved-cpus="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335525 2571 flags.go:64] FLAG: --reserved-memory="" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335529 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335532 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335535 2571 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335538 2571 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335541 2571 flags.go:64] FLAG: --runonce="false" Aug 04 22:20:57.343138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335543 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335547 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335549 2571 flags.go:64] FLAG: --seccomp-default="false" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335552 2571 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335555 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335558 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335561 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335564 2571 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335567 2571 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335570 2571 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335573 2571 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335575 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335578 2571 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335581 2571 flags.go:64] FLAG: --system-cgroups="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335584 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335589 2571 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335592 2571 flags.go:64] FLAG: --tls-cert-file="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335595 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335601 2571 flags.go:64] FLAG: --tls-min-version="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335604 2571 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335607 2571 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335610 2571 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335613 2571 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335615 2571 flags.go:64] FLAG: --v="2" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335620 2571 flags.go:64] FLAG: --version="false" Aug 04 22:20:57.343754 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335624 2571 flags.go:64] FLAG: --vmodule="" Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335627 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.335630 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335766 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335771 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335774 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335777 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335780 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335782 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335785 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335787 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335790 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335793 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335796 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335798 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335801 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335803 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335806 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335808 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335811 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.344373 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335813 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335816 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335818 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335821 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335823 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335829 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335831 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335834 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335837 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335840 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335844 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335847 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335850 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335852 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335855 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335857 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335860 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335862 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335865 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335867 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.344855 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335870 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335872 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335875 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335877 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335879 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335882 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335885 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335888 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335890 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335893 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335895 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335897 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335900 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335902 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335905 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335908 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335910 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335915 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335918 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335920 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.345376 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335923 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335926 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335928 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335931 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335933 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335936 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335938 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335941 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335943 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335946 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335949 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335951 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335953 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335956 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335958 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335961 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335964 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335966 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335969 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335971 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.345868 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335974 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335976 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335979 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335983 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335986 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335989 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335992 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335995 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.335997 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336002 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336005 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336007 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336011 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336016 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336019 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336022 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336024 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336027 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336030 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.346358 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336045 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336047 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.336050 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.336544 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.342934 2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.342947 2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.343689 2571 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.343699 2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343760 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343765 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343768 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343772 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343775 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343777 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343781 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.346824 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343783 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343786 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343789 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343791 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343794 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343796 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343799 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343802 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343805 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343807 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343810 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343812 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343815 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343818 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343820 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343823 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343825 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343828 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343831 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343833 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.347234 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343836 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343839 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343843 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343846 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343849 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343853 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343858 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343861 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343863 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343866 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343869 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343871 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343874 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343876 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343879 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343882 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343884 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343886 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343889 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.347705 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343892 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343894 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343897 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343899 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343902 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343904 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343907 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343909 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343912 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343914 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343917 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343920 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343922 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343924 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343927 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343930 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343936 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343939 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343941 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343944 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.348172 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343946 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343949 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343951 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343954 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343956 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343959 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343961 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343964 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343966 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343970 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343974 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343977 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343980 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343983 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343986 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343988 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343991 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343993 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343996 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.343999 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.348708 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344001 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344004 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344006 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344009 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344013 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344016 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344019 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344023 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344026 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344029 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344032 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344046 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344048 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.344054 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 22:20:57.349216 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344166 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344171 2571 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344174 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344177 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344179 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344182 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344184 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344187 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344190 2571 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344192 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344195 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344198 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344200 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344203 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344206 2571 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344208 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344211 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344213 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344216 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344219 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 22:20:57.349561 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344224 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344227 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344230 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344232 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344235 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344238 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344241 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344243 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344247 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344249 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344252 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344254 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344257 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344259 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344262 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344264 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344267 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344271 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344274 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344277 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 22:20:57.350107 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344280 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344283 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344285 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344288 2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344291 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344293 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344296 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344298 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344301 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344304 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344306 2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344308 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344312 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344314 2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344317 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344319 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344322 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344324 2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344327 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 22:20:57.350588 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344329 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344332 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344335 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344337 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344340 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344342 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344345 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344347 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344349 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344352 2571 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344354 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344357 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344359 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344362 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344364 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344367 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344369 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344371 2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344375 2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344379 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 22:20:57.351161 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344382 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344385 2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344388 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344391 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344394 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344396 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344399 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344402 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344404 2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344407 2571 feature_gate.go:328] unrecognized feature gate: Example Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344409 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344412 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344414 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344417 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344419 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344422 2571 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344425 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344427 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344430 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 22:20:57.351649 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:57.344432 2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.344437 2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.344534 2571 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.346420 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.347613 2571 server.go:1019] "Starting client certificate rotation" Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.347712 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.348453 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 22:20:57.352146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.349948 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 22:20:57.358213 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.358190 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 22:20:57.371083 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.371067 2571 log.go:25] "Validated CRI v1 runtime API" Aug 04 22:20:57.373468 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.373449 2571 log.go:25] "Validated CRI v1 image API" Aug 04 22:20:57.374389 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.374375 2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 22:20:57.375087 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.375073 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 22:20:57.376641 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.376614 2571 fs.go:135] Filesystem UUIDs: map[67c47193-e555-49bf-93ef-6bda0dc56f82:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8476c401-5348-4fea-beff-c699eb855e09:/dev/nvme0n1p4] Aug 04 22:20:57.376718 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.376640 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 22:20:57.381473 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.381372 2571 manager.go:216] Machine: {Timestamp:2026-08-04 22:20:57.380144382 +0000 UTC m=+0.323650604 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101323 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b93e1af08299190c01fbbfc35d73e SystemUUID:ec2b93e1-af08-2991-90c0-1fbbfc35d73e BootID:8a3e427b-4322-4744-820a-c19660b8d9ec Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:95:3b:dd:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:95:3b:dd:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:71:7c:7f:d7:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 22:20:57.381473 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.381470 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 22:20:57.381566 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.381547 2571 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 22:20:57.384269 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.384249 2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 22:20:57.384392 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.384271 2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 22:20:57.385442 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.385432 2571 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 22:20:57.385480 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.385445 2571 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 22:20:57.385480 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.385453 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 22:20:57.385998 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.385988 2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 22:20:57.386810 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.386801 2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 22:20:57.390442 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390431 2571 state_mem.go:36] "Initialized new in-memory state store" Aug 04 22:20:57.390562 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390552 2571 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 22:20:57.390590 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390580 2571 kubelet.go:486] "Attempting to sync node with API server" Aug 04 22:20:57.390616 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390590 2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 22:20:57.390643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390617 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 22:20:57.390643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390626 2571 kubelet.go:398] "Adding apiserver pod source" Aug 04 22:20:57.390643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390635 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 22:20:57.390973 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390960 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 22:20:57.391028 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.390985 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 22:20:57.393322 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.393307 2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 22:20:57.395078 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.395063 2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 22:20:57.395168 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.395102 2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 22:20:57.395714 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.395690 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xjk2b" Aug 04 22:20:57.396282 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396269 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396290 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396299 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396307 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396319 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396327 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396338 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 22:20:57.396346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396347 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 22:20:57.396569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396357 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 22:20:57.396569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396366 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 22:20:57.396569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396386 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 22:20:57.396569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.396394 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 22:20:57.397051 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.397028 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 22:20:57.397112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.397055 2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 22:20:57.400117 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.400096 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 22:20:57.400296 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.400283 2571 server.go:1268] "Started kubelet" Aug 04 22:20:57.400363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.400345 2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 22:20:57.403859 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.403829 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 22:20:57.404132 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.404108 2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 22:20:57.404445 ip-10-0-137-148 systemd[1]: Started Kubernetes Kubelet. Aug 04 22:20:57.404644 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.404522 2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 22:20:57.404644 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.404575 2571 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 22:20:57.406429 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.406219 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xjk2b" Aug 04 22:20:57.407903 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.407888 2571 server.go:310] "Adding debug handlers to kubelet server" Aug 04 22:20:57.407903 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.407900 2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 22:20:57.408938 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.408921 2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 22:20:57.409447 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.408484 2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-148.ec2.internal.18c8ba44ea162078 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-148.ec2.internal,UID:ip-10-0-137-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-148.ec2.internal,},FirstTimestamp:2026-08-04 22:20:57.400262776 +0000 UTC m=+0.343768998,LastTimestamp:2026-08-04 22:20:57.400262776 +0000 UTC m=+0.343768998,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-148.ec2.internal,}" Aug 04 22:20:57.412451 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.412430 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 22:20:57.412451 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.412448 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 22:20:57.412604 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.412547 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.413013 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.412989 2571 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 22:20:57.413013 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413014 2571 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 22:20:57.413184 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.412989 2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 22:20:57.413184 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413163 2571 factory.go:55] Registering systemd factory Aug 04 22:20:57.413184 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413178 2571 factory.go:223] Registration of the systemd container factory successfully Aug 04 22:20:57.413311 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413208 2571 reconstruct.go:97] "Volume reconstruction finished" Aug 04 22:20:57.413311 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413217 2571 reconciler.go:29] "Reconciler: start to sync state" Aug 04 22:20:57.413382 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413374 2571 factory.go:153] Registering CRI-O factory Aug 04 22:20:57.413424 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413386 2571 factory.go:223] Registration of the crio container factory successfully Aug 04 22:20:57.413469 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413432 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 22:20:57.413469 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413454 2571 factory.go:103] Registering Raw factory Aug 04 22:20:57.413469 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413468 2571 manager.go:1196] Started watching for new ooms in manager Aug 04 22:20:57.413917 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.413905 2571 manager.go:349] Starting recovery of all containers Aug 04 22:20:57.422232 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.422213 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:57.423877 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.423860 2571 manager.go:354] Recovery completed Aug 04 22:20:57.424857 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.424838 2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.428086 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.428073 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.429695 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.429680 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.429789 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.429710 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.429789 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.429725 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.430225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.430209 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 22:20:57.430225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.430220 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 22:20:57.430329 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.430235 2571 state_mem.go:36] "Initialized new in-memory state store" Aug 04 22:20:57.430329 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.430222 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.431309 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431294 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.431309 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431319 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.431428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431330 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.431428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431345 2571 policy_none.go:49] "None policy: Start" Aug 04 22:20:57.431428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431357 2571 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 22:20:57.431428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.431367 2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 22:20:57.432256 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.432246 2571 policy_none.go:47] "Start" Aug 04 22:20:57.468818 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.468699 2571 manager.go:337] "Starting Device Plugin manager" Aug 04 22:20:57.468960 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.468834 2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 22:20:57.468960 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.468846 2571 server.go:87] "Starting device plugin registration server" Aug 04 22:20:57.469086 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.469074 2571 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 22:20:57.469121 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.469087 2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 22:20:57.469219 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.469202 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 22:20:57.469306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.469290 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 22:20:57.469306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.469304 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 22:20:57.469789 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.469772 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 22:20:57.469875 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.469823 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.569500 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.569469 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.570673 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.570410 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.570673 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.570441 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.570673 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.570456 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.570673 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.570483 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.578927 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.578913 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.578977 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.578935 2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-148.ec2.internal\": node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.593471 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.593411 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.593769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.593745 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 22:20:57.595105 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.595080 2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 22:20:57.595171 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.595147 2571 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 22:20:57.595208 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.595172 2571 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 22:20:57.595251 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.595213 2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 22:20:57.597495 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.597474 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:57.698797 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.698763 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.698797 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.698788 2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal"] Aug 04 22:20:57.698957 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.698859 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.700409 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.700380 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.700409 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.700414 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.700568 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.700425 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.701626 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.701612 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.701760 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.701745 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.701825 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.701784 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.702404 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702385 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.702475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702414 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.702475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702387 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.702475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702445 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.702475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702458 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.702475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.702424 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.703722 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.703694 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.703808 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.703729 2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 22:20:57.704417 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.704402 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientMemory" Aug 04 22:20:57.704489 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.704432 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 22:20:57.704489 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.704449 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeHasSufficientPID" Aug 04 22:20:57.713855 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.713839 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.713910 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.713863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.713910 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.713880 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/697f105faa10eb4a5de45deb313e7223-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"697f105faa10eb4a5de45deb313e7223\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.729819 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.729801 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.734590 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.734574 2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-148.ec2.internal\" not found" node="ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.799190 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.799167 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:57.814562 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814533 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.814643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814575 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.814643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/697f105faa10eb4a5de45deb313e7223-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"697f105faa10eb4a5de45deb313e7223\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.814643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814611 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.814742 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814637 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c99db2fdfef28e44646e2c704f47a8ff-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal\" (UID: \"c99db2fdfef28e44646e2c704f47a8ff\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.814742 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:57.814669 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/697f105faa10eb4a5de45deb313e7223-config\") pod \"kube-apiserver-proxy-ip-10-0-137-148.ec2.internal\" (UID: \"697f105faa10eb4a5de45deb313e7223\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:57.899330 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:57.899270 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:58.005824 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.005797 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:58.033307 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.033271 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:58.038461 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.038264 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:58.106971 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.106944 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:58.212252 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.212181 2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-148.ec2.internal\" not found" Aug 04 22:20:58.295509 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.295485 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:58.312777 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.312756 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" Aug 04 22:20:58.325130 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.325110 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 22:20:58.327227 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.327213 2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" Aug 04 22:20:58.336673 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.336650 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 22:20:58.347105 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.347090 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 22:20:58.347209 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.347193 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 22:20:58.347266 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.347243 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 22:20:58.347301 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.347243 2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Aug 04 22:20:58.393942 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.393921 2571 apiserver.go:52] "Watching apiserver" Aug 04 22:20:58.410498 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.410478 2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 22:20:58.410844 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.410825 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-pdlln","kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j","openshift-cluster-node-tuning-operator/tuned-4kzhl","openshift-dns/node-resolver-m2wkk","openshift-image-registry/node-ca-wpdwv","openshift-multus/multus-56572","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal","openshift-multus/multus-additional-cni-plugins-2qkwg","openshift-multus/network-metrics-daemon-9gzwl","openshift-network-diagnostics/network-check-target-d4lpr","openshift-network-operator/iptables-alerter-pqdpm","openshift-ovn-kubernetes/ovnkube-node-jp9tc","kube-system/global-pull-secret-syncer-d6wxm"] Aug 04 22:20:58.412156 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.412129 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 22:15:58 +0000 UTC" deadline="2028-01-28 15:16:20.580071091 +0000 UTC" Aug 04 22:20:58.412206 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.412158 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13000h55m22.167916514s" Aug 04 22:20:58.412298 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.412286 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-56572" Aug 04 22:20:58.412610 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.412538 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 22:20:58.413547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.413532 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.414772 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.414734 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.414993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.414967 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 22:20:58.415245 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.415230 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.415317 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.415300 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.416347 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.415740 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 22:20:58.416347 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.415815 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-s9jb9\"" Aug 04 22:20:58.416347 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.416232 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.416588 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.416568 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.416654 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mbhs2\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.416874 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.417231 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.417247 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.417265 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.417742 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.417912 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pkcxp\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.418151 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.418268 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.418363 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.418272 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.418876 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.418858 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.419437 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-kubelet\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419437 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419441 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-systemd-units\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419458 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-slash\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419483 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-etc-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419506 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-node-log\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419549 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-socket-dir-parent\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419565 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-netns\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419600 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-etc-kubernetes\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419631 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/ca5398fd-a6b8-46ed-8891-aa72bf184169-kube-api-access-6npxv\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419695 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7k8w\" (UniqueName: \"kubernetes.io/projected/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-kube-api-access-g7k8w\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-var-lib-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.419832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419799 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-cnibin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419844 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-conf-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419871 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-daemon-config\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419895 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419896 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-log-socket\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419914 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.419997 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-systemd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-ovn\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420064 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-bin\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420089 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-netd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420112 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-cni-binary-copy\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420126 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fbld5\"" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420148 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-multus-certs\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420175 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-env-overrides\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420207 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovn-node-metrics-cert\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420224 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-system-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420250 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-bin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420266 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-multus\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420283 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-hostroot\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420301 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-kubelet\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420324 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-netns\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420353 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-config\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420375 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-os-release\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420398 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-k8s-cni-cncf-io\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.420838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420414 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-script-lib\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.421323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.420890 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.421434 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.421419 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8qv7t\"" Aug 04 22:20:58.421564 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.421550 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.421787 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.421775 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.422755 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.422740 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.422934 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.422915 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 22:20:58.423450 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.423429 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.423555 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.423435 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.423555 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.423439 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4lqvx\"" Aug 04 22:20:58.424114 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.424096 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.424576 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.424552 2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 22:20:58.425112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.425091 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 22:20:58.425112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.425111 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dprxj\"" Aug 04 22:20:58.425280 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.425265 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 22:20:58.425442 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.425423 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.425962 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.425495 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:20:58.426270 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.426256 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 22:20:58.426508 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.426495 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 22:20:58.426633 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.426618 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6z66p\"" Aug 04 22:20:58.426863 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.426848 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.426954 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.426934 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:20:58.428206 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.428070 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:20:58.428206 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.428162 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:20:58.429548 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.429528 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.432010 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.431991 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 22:20:58.432106 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.432072 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 22:20:58.432106 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.432090 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 22:20:58.432245 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.432160 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r5scw\"" Aug 04 22:20:58.445873 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.445853 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qt8x8" Aug 04 22:20:58.453737 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.453721 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qt8x8" Aug 04 22:20:58.514567 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.514542 2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 22:20:58.521467 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521446 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/340fd968-a788-4c39-b126-2cc4b93f4ffc-hosts-file\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.521553 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521476 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-system-cni-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.521553 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-os-release\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.521553 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521515 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521669 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521572 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-socket-dir-parent\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521669 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521637 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-netns\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521679 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-sys-fs\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.521744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521680 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521704 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-netns\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521717 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-socket-dir-parent\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.521744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521733 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-dbus\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521753 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521804 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-var-lib-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-host\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521854 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-var-lib-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521868 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-tmp\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521892 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-cnibin\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521914 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.521945 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7a049d9-4840-4128-a132-b445eb958f08-serviceca\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521966 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-cnibin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.521992 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522017 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysconfig\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-cnibin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522042 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522069 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522121 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522144 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-bin\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522165 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-netd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522191 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvrg6\" (UniqueName: \"kubernetes.io/projected/e06f6356-2915-4a04-b664-c78a92a4b965-kube-api-access-zvrg6\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522214 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp26t\" (UniqueName: \"kubernetes.io/projected/340fd968-a788-4c39-b126-2cc4b93f4ffc-kube-api-access-vp26t\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522221 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-bin\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522240 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-binary-copy\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-cni-netd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522272 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-cni-binary-copy\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.522323 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-multus-certs\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522417 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-env-overrides\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522450 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-multus-certs\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522498 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovn-node-metrics-cert\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522540 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-system-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522570 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-multus\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522614 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-multus\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522622 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-hostroot\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522631 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-system-cni-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-kubelet\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46fmw\" (UniqueName: \"kubernetes.io/projected/e7a049d9-4840-4128-a132-b445eb958f08-kube-api-access-46fmw\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522786 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-kubelet\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522802 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-kubelet-config\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522745 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-hostroot\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522828 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-iptables-alerter-script\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522862 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-cni-binary-copy\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522876 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-k8s-cni-cncf-io\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523023 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522886 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-env-overrides\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522905 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-socket-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vtdw\" (UniqueName: \"kubernetes.io/projected/1866f528-87ef-426c-aaf7-cdad465a6e08-kube-api-access-7vtdw\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522948 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-run-k8s-cni-cncf-io\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522954 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-kubelet\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.522977 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-systemd-units\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523000 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-slash\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523013 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523034 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-kubelet\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523047 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-systemd-units\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523050 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-slash\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-etc-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523111 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-device-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523133 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-etc-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523137 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58b2f6fe-ddca-4893-bf23-a2c844c8e419-konnectivity-ca\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-etc-kubernetes\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523192 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/ca5398fd-a6b8-46ed-8891-aa72bf184169-kube-api-access-6npxv\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523208 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.523829 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523223 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7k8w\" (UniqueName: \"kubernetes.io/projected/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-kube-api-access-g7k8w\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523265 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-openvswitch\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523277 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-etc-kubernetes\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523294 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523325 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523353 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523377 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnbz2\" (UniqueName: \"kubernetes.io/projected/f0dd7f79-567e-4af2-9787-647a72152f75-kube-api-access-hnbz2\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523395 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-ovn-kubernetes\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523400 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsrb9\" (UniqueName: \"kubernetes.io/projected/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-kube-api-access-rsrb9\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523446 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-conf-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523485 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-daemon-config\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523521 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-conf-dir\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523558 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-log-socket\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523661 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340fd968-a788-4c39-b126-2cc4b93f4ffc-tmp-dir\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523694 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-systemd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523722 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-ovn\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523666 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-log-socket\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.524781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523751 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-registration-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-systemd\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523780 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523787 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-run-ovn\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523806 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523859 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-lib-modules\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523888 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-modprobe-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-systemd\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523938 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58b2f6fe-ddca-4893-bf23-a2c844c8e419-agent-certs\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523943 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca5398fd-a6b8-46ed-8891-aa72bf184169-multus-daemon-config\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523966 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-host-slash\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.523997 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-bin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524095 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-host-var-lib-cni-bin\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524113 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-netns\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524152 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-config\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524157 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-host-run-netns\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524184 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-os-release\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.525638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524212 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-script-lib\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524256 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca5398fd-a6b8-46ed-8891-aa72bf184169-os-release\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524270 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-kubernetes\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524299 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-sys\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524345 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2x5p\" (UniqueName: \"kubernetes.io/projected/32c59839-de1f-4afe-b061-3ab933711cd5-kube-api-access-s2x5p\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524396 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a049d9-4840-4128-a132-b445eb958f08-host\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524446 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-node-log\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524517 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-conf\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524542 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-run\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-var-lib-kubelet\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524599 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-tuned\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524690 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-node-log\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524742 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-config\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.526441 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.524780 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovnkube-script-lib\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.526838 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.526707 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-ovn-node-metrics-cert\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.532806 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.532784 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6npxv\" (UniqueName: \"kubernetes.io/projected/ca5398fd-a6b8-46ed-8891-aa72bf184169-kube-api-access-6npxv\") pod \"multus-56572\" (UID: \"ca5398fd-a6b8-46ed-8891-aa72bf184169\") " pod="openshift-multus/multus-56572" Aug 04 22:20:58.533267 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.533245 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7k8w\" (UniqueName: \"kubernetes.io/projected/c2e3086f-92c3-4e6d-af6f-9f437c39b86a-kube-api-access-g7k8w\") pod \"ovnkube-node-jp9tc\" (UID: \"c2e3086f-92c3-4e6d-af6f-9f437c39b86a\") " pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.577926 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.577898 2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:58.602513 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.602472 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod697f105faa10eb4a5de45deb313e7223.slice/crio-7eedb784ea21f531ccb2d804319d0a557af811f4ef73b3141bcbe9421172d1b8 WatchSource:0}: Error finding container 7eedb784ea21f531ccb2d804319d0a557af811f4ef73b3141bcbe9421172d1b8: Status 404 returned error can't find the container with id 7eedb784ea21f531ccb2d804319d0a557af811f4ef73b3141bcbe9421172d1b8 Aug 04 22:20:58.606575 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.606557 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 22:20:58.625027 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625001 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-socket-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625045 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vtdw\" (UniqueName: \"kubernetes.io/projected/1866f528-87ef-426c-aaf7-cdad465a6e08-kube-api-access-7vtdw\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-device-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625138 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58b2f6fe-ddca-4893-bf23-a2c844c8e419-konnectivity-ca\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.625306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625153 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:20:58.625306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625180 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-device-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625239 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625298 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnbz2\" (UniqueName: \"kubernetes.io/projected/f0dd7f79-567e-4af2-9787-647a72152f75-kube-api-access-hnbz2\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625343 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsrb9\" (UniqueName: \"kubernetes.io/projected/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-kube-api-access-rsrb9\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625350 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-socket-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625390 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340fd968-a788-4c39-b126-2cc4b93f4ffc-tmp-dir\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625422 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-registration-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625441 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625456 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625474 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-lib-modules\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625506 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-registration-dir\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625532 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-etc-selinux\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625546 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-modprobe-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625583 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-systemd\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625585 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-lib-modules\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625601 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58b2f6fe-ddca-4893-bf23-a2c844c8e419-agent-certs\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625599 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.625631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625626 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-host-slash\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625647 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-systemd\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-modprobe-d\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625666 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-host-slash\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625674 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-kubernetes\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625713 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-sys\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625740 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2x5p\" (UniqueName: \"kubernetes.io/projected/32c59839-de1f-4afe-b061-3ab933711cd5-kube-api-access-s2x5p\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625766 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-kubernetes\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625769 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/58b2f6fe-ddca-4893-bf23-a2c844c8e419-konnectivity-ca\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625767 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a049d9-4840-4128-a132-b445eb958f08-host\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625788 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-sys\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625798 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/340fd968-a788-4c39-b126-2cc4b93f4ffc-tmp-dir\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625822 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-conf\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625864 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-run\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625891 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-var-lib-kubelet\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-tuned\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625919 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-run\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625940 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/340fd968-a788-4c39-b126-2cc4b93f4ffc-hosts-file\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.626369 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625955 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysctl-conf\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625967 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-system-cni-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625969 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-var-lib-kubelet\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.625994 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-os-release\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626010 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7a049d9-4840-4128-a132-b445eb958f08-host\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626018 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/340fd968-a788-4c39-b126-2cc4b93f4ffc-hosts-file\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626022 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-system-cni-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626022 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-sys-fs\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-os-release\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626107 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e06f6356-2915-4a04-b664-c78a92a4b965-sys-fs\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626117 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-dbus\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626149 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626164 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-dbus\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626178 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-host\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626199 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-tmp\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626214 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-cnibin\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626231 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.626235 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:20:58.627339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626268 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-cnibin\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7a049d9-4840-4128-a132-b445eb958f08-serviceca\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.626325 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:20:59.126303153 +0000 UTC m=+2.069809372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626354 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysconfig\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626234 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-host\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626380 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626409 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626419 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/32c59839-de1f-4afe-b061-3ab933711cd5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626435 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626465 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvrg6\" (UniqueName: \"kubernetes.io/projected/e06f6356-2915-4a04-b664-c78a92a4b965-kube-api-access-zvrg6\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626490 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp26t\" (UniqueName: \"kubernetes.io/projected/340fd968-a788-4c39-b126-2cc4b93f4ffc-kube-api-access-vp26t\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626517 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-binary-copy\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626551 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46fmw\" (UniqueName: \"kubernetes.io/projected/e7a049d9-4840-4128-a132-b445eb958f08-kube-api-access-46fmw\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626576 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-kubelet-config\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-iptables-alerter-script\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626746 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e7a049d9-4840-4128-a132-b445eb958f08-serviceca\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.626797 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:20:58.628014 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.626817 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-sysconfig\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.626903 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:20:59.126880807 +0000 UTC m=+2.070387021 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.627000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.627000 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.627078 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1d994125-f976-49cd-aff5-f27c9b651d61-kubelet-config\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.627223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-iptables-alerter-script\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.627350 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/32c59839-de1f-4afe-b061-3ab933711cd5-cni-binary-copy\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.628155 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-etc-tuned\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.628334 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/58b2f6fe-ddca-4893-bf23-a2c844c8e419-agent-certs\") pod \"konnectivity-agent-pdlln\" (UID: \"58b2f6fe-ddca-4893-bf23-a2c844c8e419\") " pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.628827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.628753 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1866f528-87ef-426c-aaf7-cdad465a6e08-tmp\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.630992 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.630974 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:20:58.630992 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.630994 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:20:58.631187 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.631004 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:20:58.631187 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:58.631064 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:20:59.131032267 +0000 UTC m=+2.074538501 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:20:58.633204 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.633176 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsrb9\" (UniqueName: \"kubernetes.io/projected/947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b-kube-api-access-rsrb9\") pod \"iptables-alerter-pqdpm\" (UID: \"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b\") " pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.633575 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.633555 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnbz2\" (UniqueName: \"kubernetes.io/projected/f0dd7f79-567e-4af2-9787-647a72152f75-kube-api-access-hnbz2\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:58.633643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.633626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vtdw\" (UniqueName: \"kubernetes.io/projected/1866f528-87ef-426c-aaf7-cdad465a6e08-kube-api-access-7vtdw\") pod \"tuned-4kzhl\" (UID: \"1866f528-87ef-426c-aaf7-cdad465a6e08\") " pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.633887 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.633868 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2x5p\" (UniqueName: \"kubernetes.io/projected/32c59839-de1f-4afe-b061-3ab933711cd5-kube-api-access-s2x5p\") pod \"multus-additional-cni-plugins-2qkwg\" (UID: \"32c59839-de1f-4afe-b061-3ab933711cd5\") " pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.634330 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.634309 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvrg6\" (UniqueName: \"kubernetes.io/projected/e06f6356-2915-4a04-b664-c78a92a4b965-kube-api-access-zvrg6\") pod \"aws-ebs-csi-driver-node-4hb9j\" (UID: \"e06f6356-2915-4a04-b664-c78a92a4b965\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.636043 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.636019 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp26t\" (UniqueName: \"kubernetes.io/projected/340fd968-a788-4c39-b126-2cc4b93f4ffc-kube-api-access-vp26t\") pod \"node-resolver-m2wkk\" (UID: \"340fd968-a788-4c39-b126-2cc4b93f4ffc\") " pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.636636 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.636618 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46fmw\" (UniqueName: \"kubernetes.io/projected/e7a049d9-4840-4128-a132-b445eb958f08-kube-api-access-46fmw\") pod \"node-ca-wpdwv\" (UID: \"e7a049d9-4840-4128-a132-b445eb958f08\") " pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.664155 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.664134 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99db2fdfef28e44646e2c704f47a8ff.slice/crio-65312346f0b7ab4aa971823ea8b4fe43117f01d65e716b71161376b1ff8b4fa8 WatchSource:0}: Error finding container 65312346f0b7ab4aa971823ea8b4fe43117f01d65e716b71161376b1ff8b4fa8: Status 404 returned error can't find the container with id 65312346f0b7ab4aa971823ea8b4fe43117f01d65e716b71161376b1ff8b4fa8 Aug 04 22:20:58.740849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.740794 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-56572" Aug 04 22:20:58.748455 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.748437 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:20:58.748650 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.748630 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5398fd_a6b8_46ed_8891_aa72bf184169.slice/crio-ba8051799bd00ee5b963c7e36347fd7b2713e6d37b886ee4c5cf99826e0bd79b WatchSource:0}: Error finding container ba8051799bd00ee5b963c7e36347fd7b2713e6d37b886ee4c5cf99826e0bd79b: Status 404 returned error can't find the container with id ba8051799bd00ee5b963c7e36347fd7b2713e6d37b886ee4c5cf99826e0bd79b Aug 04 22:20:58.756242 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.756220 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e3086f_92c3_4e6d_af6f_9f437c39b86a.slice/crio-e3b45710ccc6b225ce081777aa34feb13454cfcc5b7fe117298201f2b30c8948 WatchSource:0}: Error finding container e3b45710ccc6b225ce081777aa34feb13454cfcc5b7fe117298201f2b30c8948: Status 404 returned error can't find the container with id e3b45710ccc6b225ce081777aa34feb13454cfcc5b7fe117298201f2b30c8948 Aug 04 22:20:58.763068 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.763050 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" Aug 04 22:20:58.769222 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.769205 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" Aug 04 22:20:58.770881 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.770857 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode06f6356_2915_4a04_b664_c78a92a4b965.slice/crio-42f8271a32fbcaa6045251500807b91defac14e35b142d20ffcc9e6757fc0d81 WatchSource:0}: Error finding container 42f8271a32fbcaa6045251500807b91defac14e35b142d20ffcc9e6757fc0d81: Status 404 returned error can't find the container with id 42f8271a32fbcaa6045251500807b91defac14e35b142d20ffcc9e6757fc0d81 Aug 04 22:20:58.776898 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.776875 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1866f528_87ef_426c_aaf7_cdad465a6e08.slice/crio-7500a4b32acd27c85b7041e0ec4ca4336b928835bbfd1c29801f2bae8438b049 WatchSource:0}: Error finding container 7500a4b32acd27c85b7041e0ec4ca4336b928835bbfd1c29801f2bae8438b049: Status 404 returned error can't find the container with id 7500a4b32acd27c85b7041e0ec4ca4336b928835bbfd1c29801f2bae8438b049 Aug 04 22:20:58.781106 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.781092 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m2wkk" Aug 04 22:20:58.791053 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.791026 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340fd968_a788_4c39_b126_2cc4b93f4ffc.slice/crio-1b47cfa6fbddfe9ec9964f8a3122f673729b43394b080ca015eef6a9f09eaede WatchSource:0}: Error finding container 1b47cfa6fbddfe9ec9964f8a3122f673729b43394b080ca015eef6a9f09eaede: Status 404 returned error can't find the container with id 1b47cfa6fbddfe9ec9964f8a3122f673729b43394b080ca015eef6a9f09eaede Aug 04 22:20:58.795809 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.795791 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wpdwv" Aug 04 22:20:58.803605 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.803579 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7a049d9_4840_4128_a132_b445eb958f08.slice/crio-b74f5f516387a25386daf4c50e0d973f340653c8d45602a968ab01abb927ff1b WatchSource:0}: Error finding container b74f5f516387a25386daf4c50e0d973f340653c8d45602a968ab01abb927ff1b: Status 404 returned error can't find the container with id b74f5f516387a25386daf4c50e0d973f340653c8d45602a968ab01abb927ff1b Aug 04 22:20:58.813794 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.813779 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" Aug 04 22:20:58.821527 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.821505 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c59839_de1f_4afe_b061_3ab933711cd5.slice/crio-6b1d299654d78d3fddaf8480cc8dd5cb40a60996149032043dca5f5c84dea45c WatchSource:0}: Error finding container 6b1d299654d78d3fddaf8480cc8dd5cb40a60996149032043dca5f5c84dea45c: Status 404 returned error can't find the container with id 6b1d299654d78d3fddaf8480cc8dd5cb40a60996149032043dca5f5c84dea45c Aug 04 22:20:58.826570 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.826555 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:20:58.834193 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.834175 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b2f6fe_ddca_4893_bf23_a2c844c8e419.slice/crio-49301d3b1804118eabf4004726138679cba64e0a4936f416570a82670414fbff WatchSource:0}: Error finding container 49301d3b1804118eabf4004726138679cba64e0a4936f416570a82670414fbff: Status 404 returned error can't find the container with id 49301d3b1804118eabf4004726138679cba64e0a4936f416570a82670414fbff Aug 04 22:20:58.839832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:58.839817 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pqdpm" Aug 04 22:20:58.848484 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:20:58.848458 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947b3dd6_f736_43b1_8fc6_1b0ddbb89e4b.slice/crio-5ab8078efa6f38e71e5ed1de1d3eb1715667adb36d0c5537ee8d7d726c796606 WatchSource:0}: Error finding container 5ab8078efa6f38e71e5ed1de1d3eb1715667adb36d0c5537ee8d7d726c796606: Status 404 returned error can't find the container with id 5ab8078efa6f38e71e5ed1de1d3eb1715667adb36d0c5537ee8d7d726c796606 Aug 04 22:20:59.130651 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.130569 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:20:59.130651 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.130625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:20:59.130849 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.130745 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:20:59.130849 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.130764 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:20:59.130849 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.130829 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:00.1307862 +0000 UTC m=+3.074292415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:20:59.131001 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.130852 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:00.130840411 +0000 UTC m=+3.074346636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:20:59.231719 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.231682 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:20:59.231918 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.231888 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:20:59.231918 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.231907 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:20:59.231918 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.231920 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:20:59.232098 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.231977 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:00.231956684 +0000 UTC m=+3.175462913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:20:59.455750 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.455583 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 22:15:59 +0000 UTC" deadline="2028-02-17 19:37:32.313240398 +0000 UTC" Aug 04 22:20:59.455750 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.455648 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13485h16m32.857597779s" Aug 04 22:20:59.599804 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.598618 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:20:59.599804 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:20:59.598811 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:20:59.608288 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.608186 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" event={"ID":"1866f528-87ef-426c-aaf7-cdad465a6e08","Type":"ContainerStarted","Data":"7500a4b32acd27c85b7041e0ec4ca4336b928835bbfd1c29801f2bae8438b049"} Aug 04 22:20:59.610779 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.610725 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-56572" event={"ID":"ca5398fd-a6b8-46ed-8891-aa72bf184169","Type":"ContainerStarted","Data":"ba8051799bd00ee5b963c7e36347fd7b2713e6d37b886ee4c5cf99826e0bd79b"} Aug 04 22:20:59.613734 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.613667 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdlln" event={"ID":"58b2f6fe-ddca-4893-bf23-a2c844c8e419","Type":"ContainerStarted","Data":"49301d3b1804118eabf4004726138679cba64e0a4936f416570a82670414fbff"} Aug 04 22:20:59.620038 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.620014 2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:59.620409 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.620383 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" event={"ID":"e06f6356-2915-4a04-b664-c78a92a4b965","Type":"ContainerStarted","Data":"42f8271a32fbcaa6045251500807b91defac14e35b142d20ffcc9e6757fc0d81"} Aug 04 22:20:59.628994 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.628962 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"e3b45710ccc6b225ce081777aa34feb13454cfcc5b7fe117298201f2b30c8948"} Aug 04 22:20:59.649844 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.649500 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"c99db2fdfef28e44646e2c704f47a8ff","Type":"ContainerStarted","Data":"65312346f0b7ab4aa971823ea8b4fe43117f01d65e716b71161376b1ff8b4fa8"} Aug 04 22:20:59.674991 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.674909 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" event={"ID":"697f105faa10eb4a5de45deb313e7223","Type":"ContainerStarted","Data":"7eedb784ea21f531ccb2d804319d0a557af811f4ef73b3141bcbe9421172d1b8"} Aug 04 22:20:59.704160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.704129 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pqdpm" event={"ID":"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b","Type":"ContainerStarted","Data":"5ab8078efa6f38e71e5ed1de1d3eb1715667adb36d0c5537ee8d7d726c796606"} Aug 04 22:20:59.732395 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.732285 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerStarted","Data":"6b1d299654d78d3fddaf8480cc8dd5cb40a60996149032043dca5f5c84dea45c"} Aug 04 22:20:59.773034 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.772824 2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:20:59.781657 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.781625 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wpdwv" event={"ID":"e7a049d9-4840-4128-a132-b445eb958f08","Type":"ContainerStarted","Data":"b74f5f516387a25386daf4c50e0d973f340653c8d45602a968ab01abb927ff1b"} Aug 04 22:20:59.812898 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:20:59.812790 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2wkk" event={"ID":"340fd968-a788-4c39-b126-2cc4b93f4ffc","Type":"ContainerStarted","Data":"1b47cfa6fbddfe9ec9964f8a3122f673729b43394b080ca015eef6a9f09eaede"} Aug 04 22:21:00.114206 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.112563 2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 22:21:00.140248 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.140214 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:00.140454 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.140270 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:00.140454 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.140400 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:00.140577 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.140464 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:02.140442993 +0000 UTC m=+5.083949226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:00.140878 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.140861 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:00.140956 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.140915 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:02.140900186 +0000 UTC m=+5.084406407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:00.240759 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.240724 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:00.240933 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.240917 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:21:00.241002 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.240941 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:21:00.241002 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.240954 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:00.241135 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.241022 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:02.240995351 +0000 UTC m=+5.184501584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:00.456722 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.456580 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 22:15:59 +0000 UTC" deadline="2028-05-14 05:03:07.60915722 +0000 UTC" Aug 04 22:21:00.456722 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.456620 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15558h42m7.152541457s" Aug 04 22:21:00.596899 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.596228 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:00.596899 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:00.596250 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:00.596899 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.596400 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:00.596899 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:00.596852 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:01.598247 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:01.598215 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:01.598757 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:01.598344 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:02.155284 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:02.155247 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:02.155478 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:02.155305 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:02.155478 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.155466 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:02.155615 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.155532 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:06.155512065 +0000 UTC m=+9.099018283 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:02.155952 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.155930 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:02.156081 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.155986 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:06.155971095 +0000 UTC m=+9.099477310 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:02.256278 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:02.256239 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:02.256463 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.256400 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:21:02.256463 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.256420 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:21:02.256463 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.256432 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:02.256638 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.256492 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:06.256473587 +0000 UTC m=+9.199979820 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:02.595475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:02.595392 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:02.595635 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.595531 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:02.595635 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:02.595590 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:02.595754 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:02.595644 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:03.599429 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:03.598836 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:03.599429 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:03.598970 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:04.595453 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:04.595419 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:04.595630 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:04.595560 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:04.595992 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:04.595967 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:04.596096 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:04.596067 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:05.597922 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:05.597405 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:05.598441 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:05.598068 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:06.192265 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:06.192231 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:06.192466 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:06.192281 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:06.192466 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.192393 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:06.192466 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.192462 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:14.1924436 +0000 UTC m=+17.135949815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:06.192655 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.192393 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:06.192655 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.192567 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:14.192547681 +0000 UTC m=+17.136053897 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:06.293589 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:06.293471 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:06.293753 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.293628 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:21:06.293753 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.293649 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:21:06.293753 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.293662 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:06.293753 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.293724 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:14.293704719 +0000 UTC m=+17.237210944 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:06.596516 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:06.595885 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:06.596516 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.596036 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:06.596516 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:06.596370 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:06.596516 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:06.596460 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:07.599801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:07.599183 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:07.599801 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:07.599317 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:08.596073 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:08.596025 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:08.596254 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:08.596026 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:08.596254 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:08.596182 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:08.596369 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:08.596253 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:09.596412 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:09.596379 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:09.596857 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:09.596497 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:10.596022 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:10.595993 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:10.596193 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:10.596120 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:10.596193 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:10.596172 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:10.596291 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:10.596267 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:11.596361 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:11.596330 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:11.596832 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:11.596451 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:12.596199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:12.596165 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:12.596199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:12.596186 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:12.596406 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:12.596302 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:12.596721 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:12.596407 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:13.595992 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:13.595963 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:13.596166 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:13.596094 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:14.253384 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:14.253355 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:14.253384 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:14.253389 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:14.253891 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.253498 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:14.253891 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.253555 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:30.253540969 +0000 UTC m=+33.197047181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:14.253891 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.253498 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:14.253891 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.253625 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:30.253613651 +0000 UTC m=+33.197119880 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:14.353691 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:14.353655 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:14.353873 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.353848 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:21:14.353955 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.353886 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:21:14.353955 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.353902 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:14.354060 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.353957 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:30.353938957 +0000 UTC m=+33.297445186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:14.595761 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:14.595688 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:14.595972 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.595830 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:14.595972 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:14.595864 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:14.595972 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:14.595931 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:15.596419 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:15.596387 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:15.596852 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:15.596520 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:16.596079 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:16.596032 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:16.596256 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:16.596050 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:16.596256 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:16.596167 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:16.596378 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:16.596287 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:17.597114 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:17.597081 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:17.597571 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:17.597207 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:18.595560 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:18.595530 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:18.595719 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:18.595530 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:18.595719 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:18.595632 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:18.595816 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:18.595713 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:19.596385 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.595981 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:19.596721 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:19.596438 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:19.868343 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.868312 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-56572" event={"ID":"ca5398fd-a6b8-46ed-8891-aa72bf184169","Type":"ContainerStarted","Data":"17de1512f8853d431296353d6feddb44560e01f95f5e4ab2ed2eebef50cc26ca"} Aug 04 22:21:19.869803 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.869778 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-pdlln" event={"ID":"58b2f6fe-ddca-4893-bf23-a2c844c8e419","Type":"ContainerStarted","Data":"f50b9107b5cbe3872b33f87b6ce5cafbedd1b03c1a97cfc18860aef269cfdad3"} Aug 04 22:21:19.871653 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.871596 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" event={"ID":"e06f6356-2915-4a04-b664-c78a92a4b965","Type":"ContainerStarted","Data":"cebc8a33ad8c823c360967176ed6e837e2894476bc04e004a6d54082e9d27044"} Aug 04 22:21:19.874020 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.873997 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"4a96c7aba4e0341eb6ca145df633875c6ab872934e39eda256adc5b9d926aa93"} Aug 04 22:21:19.874129 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.874084 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"228877fda10a86123251a6cd18b953c3f3ef361e344a093d6fe75c8ae5462a91"} Aug 04 22:21:19.874168 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.874133 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"045c5049a7f1ceb3f67f8afcb212790c5c00ddb64830af2fe975075fd88b732d"} Aug 04 22:21:19.874243 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.874178 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"469ca3db62deb79cc9f6913520c15ceecee05c6481e13afa59cd2d48d7e646e5"} Aug 04 22:21:19.876507 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.876479 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" event={"ID":"697f105faa10eb4a5de45deb313e7223","Type":"ContainerStarted","Data":"7a44d4e9f7108abde323fefaf0bbdb146cacebf7a34c241a25d53134ce376614"} Aug 04 22:21:19.877586 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.877566 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerStarted","Data":"5456d42e231fcc6d8250e4b5de5cf25aa418e6b810f2bd6089fb199d57fc56c9"} Aug 04 22:21:19.878653 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.878631 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wpdwv" event={"ID":"e7a049d9-4840-4128-a132-b445eb958f08","Type":"ContainerStarted","Data":"903dc8adf41af9f40821c9184021b44340cad7e7992460ec9d7c7285ba64fe14"} Aug 04 22:21:19.879726 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.879705 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m2wkk" event={"ID":"340fd968-a788-4c39-b126-2cc4b93f4ffc","Type":"ContainerStarted","Data":"8ffd95a21fbb2cb2692d2391bbb3f96af91c09cd992f06c914b5152b3fa1ca51"} Aug 04 22:21:19.881303 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.881263 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" event={"ID":"1866f528-87ef-426c-aaf7-cdad465a6e08","Type":"ContainerStarted","Data":"6ffd4effd8bb8e69179f7c0da18b3d42c1aee05475036be66dd2de4a260e4d48"} Aug 04 22:21:19.885711 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.885651 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-56572" podStartSLOduration=1.6171139220000001 podStartE2EDuration="21.885613574s" totalImagesPullingTime="20.268499652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.885560688 +0000 UTC m=+22.829066919" watchObservedRunningTime="2026-08-04 22:21:19.885613574 +0000 UTC m=+22.829119807" Aug 04 22:21:19.905319 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.905285 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4kzhl" podStartSLOduration=1.696759635 podStartE2EDuration="21.905271371s" totalImagesPullingTime="20.208511736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.905053653 +0000 UTC m=+22.848559885" watchObservedRunningTime="2026-08-04 22:21:19.905271371 +0000 UTC m=+22.848777601" Aug 04 22:21:19.917816 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.917778 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m2wkk" podStartSLOduration=1.7247799700000002 podStartE2EDuration="21.917763641s" totalImagesPullingTime="20.192983671s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.917510054 +0000 UTC m=+22.861016285" watchObservedRunningTime="2026-08-04 22:21:19.917763641 +0000 UTC m=+22.861269872" Aug 04 22:21:19.966152 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.966106 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-148.ec2.internal" podStartSLOduration=20.966091516 podStartE2EDuration="20.966091516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.951277893 +0000 UTC m=+22.894784124" watchObservedRunningTime="2026-08-04 22:21:19.966091516 +0000 UTC m=+22.909597747" Aug 04 22:21:19.966426 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.966356 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wpdwv" podStartSLOduration=1.7862221809999999 podStartE2EDuration="21.966345137s" totalImagesPullingTime="20.180122956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.96608623 +0000 UTC m=+22.909592455" watchObservedRunningTime="2026-08-04 22:21:19.966345137 +0000 UTC m=+22.909851369" Aug 04 22:21:19.983210 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:19.983172 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-pdlln" podStartSLOduration=1.833176802 podStartE2EDuration="21.983158752s" totalImagesPullingTime="20.14998195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:19.982908094 +0000 UTC m=+22.926414324" watchObservedRunningTime="2026-08-04 22:21:19.983158752 +0000 UTC m=+22.926665036" Aug 04 22:21:20.595737 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.595706 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:20.595963 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.595705 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:20.595963 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:20.595816 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:20.595963 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:20.595899 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:20.886631 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.886435 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"114875d31389a53f69630b5976c4b542950de7882f90f56a1ba0ab02f9723708"} Aug 04 22:21:20.887422 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.886686 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"73c2f2b81bce725559bd0dd9f979ea05d9ec96ef5ae928db7713f0f9e7781eb7"} Aug 04 22:21:20.887801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.887781 2571 generic.go:356] "Generic (PLEG): container finished" podID="c99db2fdfef28e44646e2c704f47a8ff" containerID="e41427afb25dd846141a4ddc4a2d2967e0f4096a5a3024a3c81aab11ee7be0d6" exitCode=0 Aug 04 22:21:20.887879 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.887844 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"c99db2fdfef28e44646e2c704f47a8ff","Type":"ContainerDied","Data":"e41427afb25dd846141a4ddc4a2d2967e0f4096a5a3024a3c81aab11ee7be0d6"} Aug 04 22:21:20.889378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.889351 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pqdpm" event={"ID":"947b3dd6-f736-43b1-8fc6-1b0ddbb89e4b","Type":"ContainerStarted","Data":"93b10d7b7cb9d4f3d394deda713ff61e6776f0f5399559ffc86b424ac5f19106"} Aug 04 22:21:20.890789 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.890767 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="5456d42e231fcc6d8250e4b5de5cf25aa418e6b810f2bd6089fb199d57fc56c9" exitCode=0 Aug 04 22:21:20.890898 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.890865 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"5456d42e231fcc6d8250e4b5de5cf25aa418e6b810f2bd6089fb199d57fc56c9"} Aug 04 22:21:20.931914 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:20.931870 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pqdpm" podStartSLOduration=2.7968143789999997 podStartE2EDuration="22.931856223s" totalImagesPullingTime="20.135041844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:20.91296793 +0000 UTC m=+23.856474161" watchObservedRunningTime="2026-08-04 22:21:20.931856223 +0000 UTC m=+23.875362447" Aug 04 22:21:21.112696 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.112667 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 22:21:21.485196 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.485091 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T22:21:21.112685518Z","UUID":"b7ae0ae1-6cc1-457a-a96b-45052e0a2c3b","Handler":null,"Name":"","Endpoint":""} Aug 04 22:21:21.486699 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.486678 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 22:21:21.486822 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.486727 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 22:21:21.595970 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.595944 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:21.596142 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:21.596057 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:21.897581 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.897490 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" event={"ID":"e06f6356-2915-4a04-b664-c78a92a4b965","Type":"ContainerStarted","Data":"4a01035344b888721b64aa414fad0ddb96a283cbef21ac625e112302f0a4e9ea"} Aug 04 22:21:21.900310 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.900278 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" event={"ID":"c99db2fdfef28e44646e2c704f47a8ff","Type":"ContainerStarted","Data":"9f86da99a01a7cdc11af39402505af2c934181044463e7f06fadb8ff56d9e067"} Aug 04 22:21:21.914855 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:21.914809 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-148.ec2.internal" podStartSLOduration=22.914793226 podStartE2EDuration="22.914793226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:21.914025322 +0000 UTC m=+24.857531554" watchObservedRunningTime="2026-08-04 22:21:21.914793226 +0000 UTC m=+24.858299456" Aug 04 22:21:22.595698 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:22.595666 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:22.595849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:22.595670 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:22.595849 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:22.595800 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:22.595975 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:22.595868 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:22.905345 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:22.905243 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" event={"ID":"e06f6356-2915-4a04-b664-c78a92a4b965","Type":"ContainerStarted","Data":"4d8efbeb76b74be3f1a4eefc7f9b609cb2104a4bd541e1c23f67d2aa18da86c4"} Aug 04 22:21:22.909098 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:22.909069 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"78232e55cea9156e9f0c3134a32a8b216b5f440ba54d358e96ca7355716a9294"} Aug 04 22:21:22.922397 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:22.922347 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4hb9j" podStartSLOduration=2.358567271 podStartE2EDuration="24.922330604s" totalImagesPullingTime="22.563763333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:22.922223728 +0000 UTC m=+25.865729969" watchObservedRunningTime="2026-08-04 22:21:22.922330604 +0000 UTC m=+25.865836835" Aug 04 22:21:23.427674 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:23.427639 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:21:23.428289 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:23.428269 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:21:23.596332 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:23.596300 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:23.596503 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:23.596422 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:23.911654 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:23.911623 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:21:23.912136 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:23.911944 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-pdlln" Aug 04 22:21:24.595851 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:24.595784 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:24.595851 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:24.595804 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:24.596049 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:24.595914 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:24.596101 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:24.596056 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:25.595667 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.595553 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:25.596136 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:25.595740 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:25.918898 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.918818 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" event={"ID":"c2e3086f-92c3-4e6d-af6f-9f437c39b86a","Type":"ContainerStarted","Data":"201f31fb54c089af2bb19c95ba2ba168ebd3329154fc43db2e3faf5598e2b085"} Aug 04 22:21:25.919078 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.919060 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:21:25.920702 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.920678 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="c42468bb845b6bfc506186b4c9a8dbd23c61b0d03001856de3aeabb9406e855f" exitCode=0 Aug 04 22:21:25.920798 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.920747 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"c42468bb845b6bfc506186b4c9a8dbd23c61b0d03001856de3aeabb9406e855f"} Aug 04 22:21:25.935527 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.935510 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:21:25.944787 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:25.944736 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" podStartSLOduration=7.254927762 podStartE2EDuration="27.944721404s" totalImagesPullingTime="20.689793642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:25.944655555 +0000 UTC m=+28.888161786" watchObservedRunningTime="2026-08-04 22:21:25.944721404 +0000 UTC m=+28.888227629" Aug 04 22:21:26.595570 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:26.595544 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:26.595718 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:26.595674 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:26.596193 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:26.595732 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:26.596193 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:26.595811 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:26.923763 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:26.923692 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:21:26.923877 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:26.923788 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:21:26.937347 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:26.937322 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:21:27.464329 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.464160 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d6wxm"] Aug 04 22:21:27.464486 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.464429 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:27.464563 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:27.464542 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:27.467491 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.467456 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gzwl"] Aug 04 22:21:27.467609 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.467551 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:27.467656 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:27.467637 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:27.468263 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.468243 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4lpr"] Aug 04 22:21:27.468366 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.468321 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:27.468420 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:27.468390 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:27.927276 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.927243 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="83f9d7cb0a9f8fd1c099394a79033abbbcdf1ad753c55ac444e8f7d1a4ebfa1f" exitCode=0 Aug 04 22:21:27.927701 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:27.927331 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"83f9d7cb0a9f8fd1c099394a79033abbbcdf1ad753c55ac444e8f7d1a4ebfa1f"} Aug 04 22:21:29.595413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:29.595382 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:29.595413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:29.595398 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:29.595862 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:29.595399 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:29.595862 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:29.595471 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:29.595862 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:29.595554 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:29.595862 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:29.595606 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:29.933762 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:29.933686 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="5a3ce677aa5d0187ee9f9560451c113d1cf8da5d9551c700fb3dbfb7782e2cea" exitCode=0 Aug 04 22:21:29.933762 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:29.933723 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"5a3ce677aa5d0187ee9f9560451c113d1cf8da5d9551c700fb3dbfb7782e2cea"} Aug 04 22:21:30.277856 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:30.277827 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:30.277856 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:30.277862 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:30.278069 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.277969 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:30.278069 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.278021 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:22:02.278007317 +0000 UTC m=+65.221513525 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 22:21:30.278069 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.277969 2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:30.278173 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.278098 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret podName:1d994125-f976-49cd-aff5-f27c9b651d61 nodeName:}" failed. No retries permitted until 2026-08-04 22:22:02.278085506 +0000 UTC m=+65.221591719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret") pod "global-pull-secret-syncer-d6wxm" (UID: "1d994125-f976-49cd-aff5-f27c9b651d61") : object "kube-system"/"original-pull-secret" not registered Aug 04 22:21:30.378348 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:30.378308 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:30.378529 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.378486 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 22:21:30.378529 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.378511 2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 22:21:30.378529 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.378525 2571 projected.go:196] Error preparing data for projected volume kube-api-access-htdj6 for pod openshift-network-diagnostics/network-check-target-d4lpr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:30.378683 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:30.378591 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6 podName:9abc9729-68cc-424d-856e-dc41b2e93259 nodeName:}" failed. No retries permitted until 2026-08-04 22:22:02.378570768 +0000 UTC m=+65.322076997 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-htdj6" (UniqueName: "kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6") pod "network-check-target-d4lpr" (UID: "9abc9729-68cc-424d-856e-dc41b2e93259") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 22:21:31.596299 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.596267 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:31.596765 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.596267 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:31.596765 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:31.596391 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d6wxm" podUID="1d994125-f976-49cd-aff5-f27c9b651d61" Aug 04 22:21:31.596765 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:31.596510 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:21:31.596765 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.596542 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:31.596765 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:31.596613 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4lpr" podUID="9abc9729-68cc-424d-856e-dc41b2e93259" Aug 04 22:21:31.879139 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.879069 2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-148.ec2.internal" event="NodeReady" Aug 04 22:21:31.879316 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.879218 2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 22:21:31.923718 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.923687 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j2zms"] Aug 04 22:21:31.926001 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.925981 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:31.926467 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.926448 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wc957"] Aug 04 22:21:31.928282 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.928264 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:31.928385 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.928340 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d46fp\"" Aug 04 22:21:31.928385 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.928350 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 22:21:31.928493 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.928358 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 22:21:31.931060 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.930875 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qld8r\"" Aug 04 22:21:31.931060 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.930948 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 22:21:31.931060 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.930989 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 22:21:31.931264 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.931116 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 22:21:31.937413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.937377 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j2zms"] Aug 04 22:21:31.940274 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:31.940252 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wc957"] Aug 04 22:21:32.091946 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.091915 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efda0d95-7b75-435c-a1a9-e0eadf43d989-cert\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.092142 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.091962 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26sjk\" (UniqueName: \"kubernetes.io/projected/df6f5703-7447-44a9-9fb4-fcc924d73400-kube-api-access-26sjk\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.092142 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.092027 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.092142 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.092077 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv6tc\" (UniqueName: \"kubernetes.io/projected/efda0d95-7b75-435c-a1a9-e0eadf43d989-kube-api-access-qv6tc\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.092142 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.092128 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6f5703-7447-44a9-9fb4-fcc924d73400-tmp-dir\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.092338 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.092153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df6f5703-7447-44a9-9fb4-fcc924d73400-config-volume\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.192740 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192659 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6f5703-7447-44a9-9fb4-fcc924d73400-tmp-dir\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.192740 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192696 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df6f5703-7447-44a9-9fb4-fcc924d73400-config-volume\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.192953 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192746 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efda0d95-7b75-435c-a1a9-e0eadf43d989-cert\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.192953 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192774 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26sjk\" (UniqueName: \"kubernetes.io/projected/df6f5703-7447-44a9-9fb4-fcc924d73400-kube-api-access-26sjk\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.192953 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192804 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.192953 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:32.192898 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:32.193127 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.192949 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv6tc\" (UniqueName: \"kubernetes.io/projected/efda0d95-7b75-435c-a1a9-e0eadf43d989-kube-api-access-qv6tc\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.193127 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:32.193013 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:32.692992352 +0000 UTC m=+35.636498572 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:32.193127 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.193062 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df6f5703-7447-44a9-9fb4-fcc924d73400-tmp-dir\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.193352 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.193333 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/df6f5703-7447-44a9-9fb4-fcc924d73400-config-volume\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.197699 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.197587 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/efda0d95-7b75-435c-a1a9-e0eadf43d989-cert\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.203988 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.203963 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26sjk\" (UniqueName: \"kubernetes.io/projected/df6f5703-7447-44a9-9fb4-fcc924d73400-kube-api-access-26sjk\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.204180 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.204161 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv6tc\" (UniqueName: \"kubernetes.io/projected/efda0d95-7b75-435c-a1a9-e0eadf43d989-kube-api-access-qv6tc\") pod \"ingress-canary-wc957\" (UID: \"efda0d95-7b75-435c-a1a9-e0eadf43d989\") " pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.247140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.247114 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wc957" Aug 04 22:21:32.398845 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.398811 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wc957"] Aug 04 22:21:32.404071 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:21:32.404021 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefda0d95_7b75_435c_a1a9_e0eadf43d989.slice/crio-0d4a40f81c68f48679f438c9a890fb460875cc267918c6a7f8ebc50221202a28 WatchSource:0}: Error finding container 0d4a40f81c68f48679f438c9a890fb460875cc267918c6a7f8ebc50221202a28: Status 404 returned error can't find the container with id 0d4a40f81c68f48679f438c9a890fb460875cc267918c6a7f8ebc50221202a28 Aug 04 22:21:32.697187 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.697155 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:32.697833 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:32.697327 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:32.697833 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:32.697394 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:33.697374646 +0000 UTC m=+36.640880878 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:32.942573 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:32.942537 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wc957" event={"ID":"efda0d95-7b75-435c-a1a9-e0eadf43d989","Type":"ContainerStarted","Data":"0d4a40f81c68f48679f438c9a890fb460875cc267918c6a7f8ebc50221202a28"} Aug 04 22:21:33.595798 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.595764 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:21:33.595963 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.595764 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:21:33.595963 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.595954 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:21:33.599313 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599287 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5596r\"" Aug 04 22:21:33.599428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599322 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 22:21:33.599428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599383 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 22:21:33.599428 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599323 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 22:21:33.599523 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599457 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j2l9z\"" Aug 04 22:21:33.599523 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.599471 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 22:21:33.704517 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:33.704484 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:33.705087 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:33.704617 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:33.705087 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:33.704683 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:35.704665025 +0000 UTC m=+38.648171239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:34.950183 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:34.950151 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="15335c7eb49f8d5cfe37fbe4fd1a9e1b1ea4d494f999353bd929f1a4efff907c" exitCode=0 Aug 04 22:21:34.950662 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:34.950196 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"15335c7eb49f8d5cfe37fbe4fd1a9e1b1ea4d494f999353bd929f1a4efff907c"} Aug 04 22:21:35.718378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:35.718124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:35.718586 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:35.718284 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:35.718586 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:35.718551 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:39.718530301 +0000 UTC m=+42.662036512 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:35.955226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:35.955191 2571 generic.go:356] "Generic (PLEG): container finished" podID="32c59839-de1f-4afe-b061-3ab933711cd5" containerID="8f1ddd0f43f4a01c0fc11fd472ab96f39aeea21d68f0edb8599bef4c43cd8c03" exitCode=0 Aug 04 22:21:35.955563 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:35.955235 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerDied","Data":"8f1ddd0f43f4a01c0fc11fd472ab96f39aeea21d68f0edb8599bef4c43cd8c03"} Aug 04 22:21:36.960518 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:36.960485 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" event={"ID":"32c59839-de1f-4afe-b061-3ab933711cd5","Type":"ContainerStarted","Data":"a95a83a1d5478af84af9c5c1ec88ef7ac449d59f48b1d707c566bbe81538c569"} Aug 04 22:21:36.961675 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:36.961652 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wc957" event={"ID":"efda0d95-7b75-435c-a1a9-e0eadf43d989","Type":"ContainerStarted","Data":"49d3e6aa2cd837c398b8eadaebf782c756ac708692b2193647f351bc94cbf68d"} Aug 04 22:21:36.981445 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:36.981403 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2qkwg" podStartSLOduration=8.051425781 podStartE2EDuration="38.981388222s" totalImagesPullingTime="30.929962441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:36.980697039 +0000 UTC m=+39.924203359" watchObservedRunningTime="2026-08-04 22:21:36.981388222 +0000 UTC m=+39.924894447" Aug 04 22:21:36.994748 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:36.994708 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wc957" podStartSLOduration=2.298233023 podStartE2EDuration="5.994696659s" totalImagesPullingTime="3.696463636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:21:36.994092518 +0000 UTC m=+39.937598755" watchObservedRunningTime="2026-08-04 22:21:36.994696659 +0000 UTC m=+39.938203274" Aug 04 22:21:39.746474 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:39.746438 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:39.746854 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:39.746586 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:39.746854 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:39.746647 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:21:47.746631284 +0000 UTC m=+50.690137493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:47.798618 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:47.798581 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:21:47.798987 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:47.798692 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:21:47.798987 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:21:47.798748 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:22:03.798731488 +0000 UTC m=+66.742237717 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:21:58.939738 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:21:58.939712 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jp9tc" Aug 04 22:22:02.300027 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.299970 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:22:02.300027 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.300029 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:22:02.302582 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.302566 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 22:22:02.302643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.302565 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 22:22:02.310310 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:02.310288 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 04 22:22:02.310416 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:02.310372 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:23:06.310357694 +0000 UTC m=+129.253863902 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : secret "metrics-daemon-secret" not found Aug 04 22:22:02.313563 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.313543 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1d994125-f976-49cd-aff5-f27c9b651d61-original-pull-secret\") pod \"global-pull-secret-syncer-d6wxm\" (UID: \"1d994125-f976-49cd-aff5-f27c9b651d61\") " pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:22:02.400890 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.400852 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:22:02.403242 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.403226 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 22:22:02.407968 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.407955 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d6wxm" Aug 04 22:22:02.413368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.413348 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 22:22:02.424537 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.424514 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdj6\" (UniqueName: \"kubernetes.io/projected/9abc9729-68cc-424d-856e-dc41b2e93259-kube-api-access-htdj6\") pod \"network-check-target-d4lpr\" (UID: \"9abc9729-68cc-424d-856e-dc41b2e93259\") " pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:22:02.521984 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.521946 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d6wxm"] Aug 04 22:22:02.525223 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:22:02.525194 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d994125_f976_49cd_aff5_f27c9b651d61.slice/crio-519e4a6a346adf13b64c6d76d0f68ad27273adcf685a6e50fd306c8c737354e8 WatchSource:0}: Error finding container 519e4a6a346adf13b64c6d76d0f68ad27273adcf685a6e50fd306c8c737354e8: Status 404 returned error can't find the container with id 519e4a6a346adf13b64c6d76d0f68ad27273adcf685a6e50fd306c8c737354e8 Aug 04 22:22:02.722925 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.722901 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j2l9z\"" Aug 04 22:22:02.730809 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.730789 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:22:02.854707 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:02.854678 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4lpr"] Aug 04 22:22:02.857770 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:22:02.857747 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9abc9729_68cc_424d_856e_dc41b2e93259.slice/crio-23316440a672039600146fdd012278febdd1cddefd45c8d82fc9f597e2676400 WatchSource:0}: Error finding container 23316440a672039600146fdd012278febdd1cddefd45c8d82fc9f597e2676400: Status 404 returned error can't find the container with id 23316440a672039600146fdd012278febdd1cddefd45c8d82fc9f597e2676400 Aug 04 22:22:03.018719 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:03.018634 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4lpr" event={"ID":"9abc9729-68cc-424d-856e-dc41b2e93259","Type":"ContainerStarted","Data":"23316440a672039600146fdd012278febdd1cddefd45c8d82fc9f597e2676400"} Aug 04 22:22:03.019955 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:03.019922 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d6wxm" event={"ID":"1d994125-f976-49cd-aff5-f27c9b651d61","Type":"ContainerStarted","Data":"519e4a6a346adf13b64c6d76d0f68ad27273adcf685a6e50fd306c8c737354e8"} Aug 04 22:22:03.822682 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:03.822646 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:22:03.823295 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:03.822803 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:22:03.823295 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:03.822878 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:22:35.822856662 +0000 UTC m=+98.766363068 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:22:08.035816 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:08.035777 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4lpr" event={"ID":"9abc9729-68cc-424d-856e-dc41b2e93259","Type":"ContainerStarted","Data":"fd5007bb6a321dfc95fe543735de0221d528c3fe29ee2abde6ba1209f9eb66ba"} Aug 04 22:22:08.036245 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:08.035907 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:22:08.037231 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:08.037208 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d6wxm" event={"ID":"1d994125-f976-49cd-aff5-f27c9b651d61","Type":"ContainerStarted","Data":"fcec4ff667a24bf734c4353480c74e4e8c95a8ac81254c183cf9c1160cfbb217"} Aug 04 22:22:08.049475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:08.049435 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d4lpr" podStartSLOduration=65.555317359 podStartE2EDuration="1m10.049423226s" totalImagesPullingTime="4.494105867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:22:08.049116066 +0000 UTC m=+70.992622298" watchObservedRunningTime="2026-08-04 22:22:08.049423226 +0000 UTC m=+70.992929457" Aug 04 22:22:08.061693 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:08.061648 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d6wxm" podStartSLOduration=65.23098886 podStartE2EDuration="1m10.061632954s" totalImagesPullingTime="4.830644094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:22:08.061434887 +0000 UTC m=+71.004941119" watchObservedRunningTime="2026-08-04 22:22:08.061632954 +0000 UTC m=+71.005139185" Aug 04 22:22:23.011943 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.011804 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7"] Aug 04 22:22:23.015686 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.015657 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.015686 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.015678 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v"] Aug 04 22:22:23.017402 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.017385 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.017786 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.017766 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 22:22:23.017872 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.017855 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4l9p9\"" Aug 04 22:22:23.018049 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.018016 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 22:22:23.018109 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.018021 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 22:22:23.018562 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.018548 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Aug 04 22:22:23.019480 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.019460 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Aug 04 22:22:23.019568 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.019466 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Aug 04 22:22:23.019568 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.019464 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Aug 04 22:22:23.019747 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.019731 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Aug 04 22:22:23.025450 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.025431 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7"] Aug 04 22:22:23.026773 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.026740 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v"] Aug 04 22:22:23.149226 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149202 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.149346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149232 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19d1da-1437-4d92-9559-f6493cc65580-additional-service-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.149346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149252 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.149346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149288 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzzzd\" (UniqueName: \"kubernetes.io/projected/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-kube-api-access-jzzzd\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.149346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149308 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.149346 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149328 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.149560 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149359 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.149560 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.149428 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnmzn\" (UniqueName: \"kubernetes.io/projected/4e19d1da-1437-4d92-9559-f6493cc65580-kube-api-access-wnmzn\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250489 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250464 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzzzd\" (UniqueName: \"kubernetes.io/projected/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-kube-api-access-jzzzd\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.250563 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250492 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250563 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250510 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250563 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.250710 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250568 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnmzn\" (UniqueName: \"kubernetes.io/projected/4e19d1da-1437-4d92-9559-f6493cc65580-kube-api-access-wnmzn\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250710 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250677 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250808 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250715 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19d1da-1437-4d92-9559-f6493cc65580-additional-service-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.250808 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.250743 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.251454 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.251426 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e19d1da-1437-4d92-9559-f6493cc65580-additional-service-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.253141 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.253112 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.253225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.253162 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-ca\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.253225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.253181 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.253484 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.253468 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.253519 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.253500 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4e19d1da-1437-4d92-9559-f6493cc65580-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.257917 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.257893 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnmzn\" (UniqueName: \"kubernetes.io/projected/4e19d1da-1437-4d92-9559-f6493cc65580-kube-api-access-wnmzn\") pod \"cluster-proxy-proxy-agent-559d964d76-h7b7v\" (UID: \"4e19d1da-1437-4d92-9559-f6493cc65580\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.258174 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.258153 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzzzd\" (UniqueName: \"kubernetes.io/projected/4c8a5deb-e1b6-47d8-992b-08f036b3ce6d-kube-api-access-jzzzd\") pod \"managed-serviceaccount-addon-agent-7c648db48b-h8qq7\" (UID: \"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.332523 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.332471 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" Aug 04 22:22:23.337892 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.337874 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:22:23.457768 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.457743 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7"] Aug 04 22:22:23.460602 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:22:23.460577 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c8a5deb_e1b6_47d8_992b_08f036b3ce6d.slice/crio-899ac3393aa93b2e0ba293a20f6061a718c4fff937a0d8e91d45f6d159eaf1a7 WatchSource:0}: Error finding container 899ac3393aa93b2e0ba293a20f6061a718c4fff937a0d8e91d45f6d159eaf1a7: Status 404 returned error can't find the container with id 899ac3393aa93b2e0ba293a20f6061a718c4fff937a0d8e91d45f6d159eaf1a7 Aug 04 22:22:23.471787 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:23.471672 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v"] Aug 04 22:22:23.476085 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:22:23.476060 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e19d1da_1437_4d92_9559_f6493cc65580.slice/crio-23fa3ee7dc07cd42a76b40882e2ac2b56c24b3096d0033fbe99acb1e92d9f88f WatchSource:0}: Error finding container 23fa3ee7dc07cd42a76b40882e2ac2b56c24b3096d0033fbe99acb1e92d9f88f: Status 404 returned error can't find the container with id 23fa3ee7dc07cd42a76b40882e2ac2b56c24b3096d0033fbe99acb1e92d9f88f Aug 04 22:22:24.074598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:24.074555 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerStarted","Data":"23fa3ee7dc07cd42a76b40882e2ac2b56c24b3096d0033fbe99acb1e92d9f88f"} Aug 04 22:22:24.075528 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:24.075506 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" event={"ID":"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d","Type":"ContainerStarted","Data":"899ac3393aa93b2e0ba293a20f6061a718c4fff937a0d8e91d45f6d159eaf1a7"} Aug 04 22:22:29.092547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.092510 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerStarted","Data":"7232fe9ae564537e47bb980049abf5294a9daecc7dd759fb18d7d5612732a529"} Aug 04 22:22:29.093137 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.092556 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerStarted","Data":"b292fdf687bda73bd3a97d7d944eb0d48290452bad303419f5d2cb6e3dff754e"} Aug 04 22:22:29.093137 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.092574 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerStarted","Data":"1b99baa73ded54568dae57ba5d3385c429c2d53f2e124851732db6edbbf2c133"} Aug 04 22:22:29.093835 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.093807 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" event={"ID":"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d","Type":"ContainerStarted","Data":"253594680fc95c7bb56c75117fd7d75ce04620c7b0fb33e03b20acd115899c57"} Aug 04 22:22:29.123057 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.122998 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" podStartSLOduration=2.403006446 podStartE2EDuration="7.122984976s" totalImagesPullingTime="4.71997853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:22:29.10918648 +0000 UTC m=+92.052692710" watchObservedRunningTime="2026-08-04 22:22:29.122984976 +0000 UTC m=+92.066491206" Aug 04 22:22:29.123348 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:29.123320 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" podStartSLOduration=2.452820289 podStartE2EDuration="7.123314074s" totalImagesPullingTime="4.670493785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:22:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:22:29.122393574 +0000 UTC m=+92.065899805" watchObservedRunningTime="2026-08-04 22:22:29.123314074 +0000 UTC m=+92.066820305" Aug 04 22:22:35.842137 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:35.842098 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:22:35.842538 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:35.842210 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:22:35.842538 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:22:35.842270 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:23:39.842257557 +0000 UTC m=+162.785763767 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:22:39.042134 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:22:39.042099 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d4lpr" Aug 04 22:23:06.347549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:23:06.347503 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:23:06.348031 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:06.347634 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 04 22:23:06.348031 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:06.347690 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:08.347674738 +0000 UTC m=+251.291180947 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : secret "metrics-daemon-secret" not found Aug 04 22:23:34.939285 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:34.939240 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j2zms" podUID="df6f5703-7447-44a9-9fb4-fcc924d73400" Aug 04 22:23:35.271378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:23:35.271351 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2zms" Aug 04 22:23:36.614939 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:36.614897 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:23:39.872634 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:23:39.872602 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:23:39.873076 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:39.872732 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 22:23:39.873076 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:23:39.872793 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls podName:df6f5703-7447-44a9-9fb4-fcc924d73400 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:41.872779059 +0000 UTC m=+284.816285268 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls") pod "dns-default-j2zms" (UID: "df6f5703-7447-44a9-9fb4-fcc924d73400") : secret "dns-default-metrics-tls" not found Aug 04 22:23:48.596007 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:23:48.595960 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:24:28.417962 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:24:28.417927 2571 generic.go:356] "Generic (PLEG): container finished" podID="4c8a5deb-e1b6-47d8-992b-08f036b3ce6d" containerID="253594680fc95c7bb56c75117fd7d75ce04620c7b0fb33e03b20acd115899c57" exitCode=255 Aug 04 22:24:28.418353 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:24:28.417979 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" event={"ID":"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d","Type":"ContainerDied","Data":"253594680fc95c7bb56c75117fd7d75ce04620c7b0fb33e03b20acd115899c57"} Aug 04 22:24:28.418353 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:24:28.418271 2571 scope.go:117] "RemoveContainer" containerID="253594680fc95c7bb56c75117fd7d75ce04620c7b0fb33e03b20acd115899c57" Aug 04 22:24:29.422608 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:24:29.422572 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7c648db48b-h8qq7" event={"ID":"4c8a5deb-e1b6-47d8-992b-08f036b3ce6d","Type":"ContainerStarted","Data":"070c5caa22cbecbc31c225d3c0246d88031486e26bc8f47d61cf2c4084ef0630"} Aug 04 22:25:08.381777 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:08.381731 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:25:08.382267 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:08.381865 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Aug 04 22:25:08.382267 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:08.381921 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs podName:f0dd7f79-567e-4af2-9787-647a72152f75 nodeName:}" failed. No retries permitted until 2026-08-04 22:27:10.381906479 +0000 UTC m=+373.325412691 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs") pod "network-metrics-daemon-9gzwl" (UID: "f0dd7f79-567e-4af2-9787-647a72152f75") : secret "metrics-daemon-secret" not found Aug 04 22:25:26.651726 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.651655 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pnlc9"] Aug 04 22:25:26.654936 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.654920 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.657225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.657204 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 22:25:26.657225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.657206 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 22:25:26.657373 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.657241 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bbwrp\"" Aug 04 22:25:26.657934 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.657918 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 22:25:26.658012 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.657922 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 22:25:26.665964 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.665943 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pnlc9"] Aug 04 22:25:26.705475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.705453 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krt95\" (UniqueName: \"kubernetes.io/projected/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-api-access-krt95\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.705573 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.705483 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-crio-socket\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.705573 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.705502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-data-volume\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.705573 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.705524 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.705690 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.705598 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806266 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krt95\" (UniqueName: \"kubernetes.io/projected/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-api-access-krt95\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806404 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806302 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-crio-socket\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806404 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-data-volume\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806499 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806407 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-crio-socket\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806499 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806436 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806499 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806476 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806610 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:26.806549 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 22:25:26.806610 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:26.806570 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 22:25:26.806610 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.806600 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-data-volume\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:26.806747 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:26.806615 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:27.306602899 +0000 UTC m=+270.250109107 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : secret "insights-runtime-extractor-tls" not found Aug 04 22:25:26.806747 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:26.806635 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:27.306629451 +0000 UTC m=+270.250135702 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : configmap "kube-rbac-proxy" not found Aug 04 22:25:26.816306 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:26.816286 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krt95\" (UniqueName: \"kubernetes.io/projected/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-api-access-krt95\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:27.310507 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:27.310468 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:27.310683 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:27.310518 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:27.310683 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:27.310581 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 22:25:27.310683 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:27.310629 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 22:25:27.310683 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:27.310652 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:28.310636979 +0000 UTC m=+271.254143188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : configmap "kube-rbac-proxy" not found Aug 04 22:25:27.310683 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:27.310670 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:28.310659649 +0000 UTC m=+271.254165858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : secret "insights-runtime-extractor-tls" not found Aug 04 22:25:28.317654 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:28.317615 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:28.318057 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:28.317665 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:28.318057 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:28.317735 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 22:25:28.318057 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:28.317763 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 22:25:28.318057 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:28.317807 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:30.317789931 +0000 UTC m=+273.261296140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : configmap "kube-rbac-proxy" not found Aug 04 22:25:28.318057 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:28.317821 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:30.317814587 +0000 UTC m=+273.261320796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : secret "insights-runtime-extractor-tls" not found Aug 04 22:25:29.219810 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:29.219787 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2wkk_340fd968-a788-4c39-b126-2cc4b93f4ffc/dns-node-resolver/0.log" Aug 04 22:25:30.332224 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:30.332185 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:30.332638 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:30.332238 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:30.332638 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:30.332304 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 22:25:30.332638 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:30.332366 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:34.332350597 +0000 UTC m=+277.275856806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : configmap "kube-rbac-proxy" not found Aug 04 22:25:30.332638 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:30.332377 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 22:25:30.332638 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:30.332429 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:34.332414766 +0000 UTC m=+277.275920975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : secret "insights-runtime-extractor-tls" not found Aug 04 22:25:30.420073 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:30.420049 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wpdwv_e7a049d9-4840-4128-a132-b445eb958f08/node-ca/0.log" Aug 04 22:25:31.419837 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:31.419810 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wc957_efda0d95-7b75-435c-a1a9-e0eadf43d989/serve-healthcheck-canary/0.log" Aug 04 22:25:34.358732 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:34.358703 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:34.359115 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:34.358741 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:34.359115 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:34.358817 2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 22:25:34.359115 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:34.358849 2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 22:25:34.359115 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:34.358882 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:42.358866573 +0000 UTC m=+285.302372790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : configmap "kube-rbac-proxy" not found Aug 04 22:25:34.359115 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:34.358898 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls podName:1fa036cc-7ce3-4b4c-992d-ff5982e62b06 nodeName:}" failed. No retries permitted until 2026-08-04 22:25:42.358890734 +0000 UTC m=+285.302396943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pnlc9" (UID: "1fa036cc-7ce3-4b4c-992d-ff5982e62b06") : secret "insights-runtime-extractor-tls" not found Aug 04 22:25:38.272619 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:38.272580 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-j2zms" podUID="df6f5703-7447-44a9-9fb4-fcc924d73400" Aug 04 22:25:38.612824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:38.612753 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:41.911766 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:41.911728 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:41.914020 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:41.913998 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/df6f5703-7447-44a9-9fb4-fcc924d73400-metrics-tls\") pod \"dns-default-j2zms\" (UID: \"df6f5703-7447-44a9-9fb4-fcc924d73400\") " pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:42.215196 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.215171 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-d46fp\"" Aug 04 22:25:42.223824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.223807 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:42.342714 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.342685 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j2zms"] Aug 04 22:25:42.345333 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:25:42.345304 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf6f5703_7447_44a9_9fb4_fcc924d73400.slice/crio-7477ee0fd3e2da4098e7d369668b1780e3babbc3c5fa29e99f9d238c7bd759e9 WatchSource:0}: Error finding container 7477ee0fd3e2da4098e7d369668b1780e3babbc3c5fa29e99f9d238c7bd759e9: Status 404 returned error can't find the container with id 7477ee0fd3e2da4098e7d369668b1780e3babbc3c5fa29e99f9d238c7bd759e9 Aug 04 22:25:42.414832 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.414808 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:42.414940 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.414842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:42.415307 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.415288 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:42.417015 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.416994 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1fa036cc-7ce3-4b4c-992d-ff5982e62b06-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pnlc9\" (UID: \"1fa036cc-7ce3-4b4c-992d-ff5982e62b06\") " pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:42.563328 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.563254 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pnlc9" Aug 04 22:25:42.624860 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.624823 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2zms" event={"ID":"df6f5703-7447-44a9-9fb4-fcc924d73400","Type":"ContainerStarted","Data":"7477ee0fd3e2da4098e7d369668b1780e3babbc3c5fa29e99f9d238c7bd759e9"} Aug 04 22:25:42.681932 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:42.681904 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pnlc9"] Aug 04 22:25:42.685020 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:25:42.684995 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa036cc_7ce3_4b4c_992d_ff5982e62b06.slice/crio-b651882685aa6801858bc92d126752fe7bcabeea869174abca1eaecedb9c8759 WatchSource:0}: Error finding container b651882685aa6801858bc92d126752fe7bcabeea869174abca1eaecedb9c8759: Status 404 returned error can't find the container with id b651882685aa6801858bc92d126752fe7bcabeea869174abca1eaecedb9c8759 Aug 04 22:25:43.628681 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:43.628648 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pnlc9" event={"ID":"1fa036cc-7ce3-4b4c-992d-ff5982e62b06","Type":"ContainerStarted","Data":"605eed9c2be181e1fd816ca516a8cbcf3a601c7a4a8c0f0173b37588f8833013"} Aug 04 22:25:43.629027 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:43.628701 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pnlc9" event={"ID":"1fa036cc-7ce3-4b4c-992d-ff5982e62b06","Type":"ContainerStarted","Data":"60c9b6f4e5b72b32177fbd99afd0db7f59b0d529e8cc73a811b67b379b26bd58"} Aug 04 22:25:43.629027 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:43.628720 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pnlc9" event={"ID":"1fa036cc-7ce3-4b4c-992d-ff5982e62b06","Type":"ContainerStarted","Data":"b651882685aa6801858bc92d126752fe7bcabeea869174abca1eaecedb9c8759"} Aug 04 22:25:44.633758 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:44.633682 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2zms" event={"ID":"df6f5703-7447-44a9-9fb4-fcc924d73400","Type":"ContainerStarted","Data":"388bd9c05748833ab17ceb1c5bbb06c335b948263356330d50901e9ca0d23e26"} Aug 04 22:25:44.633758 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:44.633736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j2zms" event={"ID":"df6f5703-7447-44a9-9fb4-fcc924d73400","Type":"ContainerStarted","Data":"467cebc681fd752871a76c8ad637b11e772c3f09f82cdbd9b87ce9d025bd537a"} Aug 04 22:25:44.634191 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:44.633832 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:44.650617 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:44.650568 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-j2zms" podStartSLOduration=251.694878367 podStartE2EDuration="4m13.650549605s" totalImagesPullingTime="1.955671238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:25:44.649430693 +0000 UTC m=+287.592936944" watchObservedRunningTime="2026-08-04 22:25:44.650549605 +0000 UTC m=+287.594055838" Aug 04 22:25:45.637679 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:45.637640 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pnlc9" event={"ID":"1fa036cc-7ce3-4b4c-992d-ff5982e62b06","Type":"ContainerStarted","Data":"7309b790d0d19aff1328ac82334cc25a9b5f040153c04c9c3042fb18512dc604"} Aug 04 22:25:47.168767 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.168715 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pnlc9" podStartSLOduration=18.804307878 podStartE2EDuration="21.168698315s" totalImagesPullingTime="2.364390437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 22:25:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:25:45.654183906 +0000 UTC m=+288.597690137" watchObservedRunningTime="2026-08-04 22:25:47.168698315 +0000 UTC m=+290.112204545" Aug 04 22:25:47.169651 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.169630 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dcbff56cb-7jlgb"] Aug 04 22:25:47.172492 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.172473 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.175115 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.175089 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 22:25:47.175203 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.175120 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 22:25:47.175203 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.175158 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 22:25:47.175280 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.175251 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h7b97\"" Aug 04 22:25:47.180008 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.179987 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 22:25:47.187431 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.187412 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dcbff56cb-7jlgb"] Aug 04 22:25:47.246752 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-bound-sa-token\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.246851 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246762 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-ca-trust-extracted\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.246851 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-trusted-ca\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.246851 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246834 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-image-registry-private-configuration\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.247005 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246872 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-installation-pull-secrets\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.247005 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.246916 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lh2l\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-kube-api-access-6lh2l\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.247144 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.247023 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-tls\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.247144 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.247083 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-certificates\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348177 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348154 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lh2l\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-kube-api-access-6lh2l\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348272 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348205 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-tls\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348272 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-certificates\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348280 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-bound-sa-token\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348321 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-ca-trust-extracted\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348341 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-trusted-ca\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348508 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348384 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-image-registry-private-configuration\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348508 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348415 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-installation-pull-secrets\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.348781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.348763 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-ca-trust-extracted\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.349145 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.349126 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-certificates\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.349465 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.349438 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-trusted-ca\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.350702 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.350674 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-registry-tls\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.350801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.350783 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-image-registry-private-configuration\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.350942 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.350922 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-installation-pull-secrets\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.364796 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.364775 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-bound-sa-token\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.367008 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.366992 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lh2l\" (UniqueName: \"kubernetes.io/projected/b19748d2-e482-4ad6-9e8e-8aaa463e0c20-kube-api-access-6lh2l\") pod \"image-registry-dcbff56cb-7jlgb\" (UID: \"b19748d2-e482-4ad6-9e8e-8aaa463e0c20\") " pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.481875 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.481850 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:47.604007 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.603975 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dcbff56cb-7jlgb"] Aug 04 22:25:47.606685 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:25:47.606663 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19748d2_e482_4ad6_9e8e_8aaa463e0c20.slice/crio-e37faa55c14b28340716e66beb20dc70952a66958da6816b8bfa75e216f96434 WatchSource:0}: Error finding container e37faa55c14b28340716e66beb20dc70952a66958da6816b8bfa75e216f96434: Status 404 returned error can't find the container with id e37faa55c14b28340716e66beb20dc70952a66958da6816b8bfa75e216f96434 Aug 04 22:25:47.645590 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:47.645561 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" event={"ID":"b19748d2-e482-4ad6-9e8e-8aaa463e0c20","Type":"ContainerStarted","Data":"e37faa55c14b28340716e66beb20dc70952a66958da6816b8bfa75e216f96434"} Aug 04 22:25:48.650194 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:48.650165 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" event={"ID":"b19748d2-e482-4ad6-9e8e-8aaa463e0c20","Type":"ContainerStarted","Data":"860a2dedf9886eab7195cafc02791be02200c0f922aaf6ce23a84694cd6b8305"} Aug 04 22:25:48.650639 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:48.650289 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:25:48.668943 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:48.668886 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" podStartSLOduration=1.6688707470000002 podStartE2EDuration="1.668870747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:25:48.66798118 +0000 UTC m=+291.611487424" watchObservedRunningTime="2026-08-04 22:25:48.668870747 +0000 UTC m=+291.612376979" Aug 04 22:25:51.596764 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:25:51.596729 2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-9gzwl" podUID="f0dd7f79-567e-4af2-9787-647a72152f75" Aug 04 22:25:54.639733 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:54.639706 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j2zms" Aug 04 22:25:57.429393 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:25:57.429370 2571 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 22:26:03.021259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.021228 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-759bp"] Aug 04 22:26:03.024583 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.024569 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.026515 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.026479 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 22:26:03.026646 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.026633 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hmr7p\"" Aug 04 22:26:03.027339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.027320 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 22:26:03.027339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.027332 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 22:26:03.027339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.027333 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 22:26:03.027559 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.027368 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 22:26:03.027559 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.027429 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 22:26:03.159963 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.159928 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.159963 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.159967 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-root\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160024 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-wtmp\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160068 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgb9\" (UniqueName: \"kubernetes.io/projected/d2e56019-3d66-4892-bb07-927365b8cb05-kube-api-access-kfgb9\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160092 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160121 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-metrics-client-ca\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160140 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-textfile\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-sys\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.160404 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.160205 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-accelerators-collector-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261394 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261362 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-root\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261400 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-wtmp\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261416 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgb9\" (UniqueName: \"kubernetes.io/projected/d2e56019-3d66-4892-bb07-927365b8cb05-kube-api-access-kfgb9\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261435 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261486 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-root\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261549 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261539 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-wtmp\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261552 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-metrics-client-ca\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261591 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-textfile\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261618 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-sys\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261660 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-accelerators-collector-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261706 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2e56019-3d66-4892-bb07-927365b8cb05-sys\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.261769 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261714 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.262076 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:26:03.261795 2571 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Aug 04 22:26:03.262076 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:26:03.261898 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls podName:d2e56019-3d66-4892-bb07-927365b8cb05 nodeName:}" failed. No retries permitted until 2026-08-04 22:26:03.761876946 +0000 UTC m=+306.705383158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls") pod "node-exporter-759bp" (UID: "d2e56019-3d66-4892-bb07-927365b8cb05") : secret "node-exporter-tls" not found Aug 04 22:26:03.262076 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.261950 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-textfile\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.262199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.262109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-metrics-client-ca\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.262234 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.262212 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-accelerators-collector-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.263621 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.263604 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.269337 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.269317 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgb9\" (UniqueName: \"kubernetes.io/projected/d2e56019-3d66-4892-bb07-927365b8cb05-kube-api-access-kfgb9\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.595773 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.595742 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:26:03.765301 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.765267 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.767480 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.767457 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d2e56019-3d66-4892-bb07-927365b8cb05-node-exporter-tls\") pod \"node-exporter-759bp\" (UID: \"d2e56019-3d66-4892-bb07-927365b8cb05\") " pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.933738 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.933657 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-759bp" Aug 04 22:26:03.943552 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:26:03.943526 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2e56019_3d66_4892_bb07_927365b8cb05.slice/crio-f743c56ae9499a7cd7efae3cf06a7635df95810a67348697380e5e834f5e1e4f WatchSource:0}: Error finding container f743c56ae9499a7cd7efae3cf06a7635df95810a67348697380e5e834f5e1e4f: Status 404 returned error can't find the container with id f743c56ae9499a7cd7efae3cf06a7635df95810a67348697380e5e834f5e1e4f Aug 04 22:26:03.945233 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:03.945214 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 22:26:04.699121 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:04.699079 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-759bp" event={"ID":"d2e56019-3d66-4892-bb07-927365b8cb05","Type":"ContainerStarted","Data":"f743c56ae9499a7cd7efae3cf06a7635df95810a67348697380e5e834f5e1e4f"} Aug 04 22:26:05.703680 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:05.703642 2571 generic.go:356] "Generic (PLEG): container finished" podID="d2e56019-3d66-4892-bb07-927365b8cb05" containerID="2f54090d3465fe2ab6ce981f1a4643f4603567e178e70529ab333be1a2e733fa" exitCode=0 Aug 04 22:26:05.704082 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:05.703715 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-759bp" event={"ID":"d2e56019-3d66-4892-bb07-927365b8cb05","Type":"ContainerDied","Data":"2f54090d3465fe2ab6ce981f1a4643f4603567e178e70529ab333be1a2e733fa"} Aug 04 22:26:06.709183 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:06.709144 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-759bp" event={"ID":"d2e56019-3d66-4892-bb07-927365b8cb05","Type":"ContainerStarted","Data":"e9fa349664daa9b220922562934c38d1c12831563ec7826f62bbd4bc1cff3361"} Aug 04 22:26:06.709602 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:06.709209 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-759bp" event={"ID":"d2e56019-3d66-4892-bb07-927365b8cb05","Type":"ContainerStarted","Data":"b48b6a894bc014e309df329f15d003386104ad439f7baec2bfa7dc40905ef525"} Aug 04 22:26:06.730738 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:06.730687 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-759bp" podStartSLOduration=3.730137358 podStartE2EDuration="4.730673441s" totalImagesPullingTime="939.347478ms" totalInitContainerRuntime="61.188605ms" isStatefulPod=true podCreationTimestamp="2026-08-04 22:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:26:06.729131705 +0000 UTC m=+309.672637935" watchObservedRunningTime="2026-08-04 22:26:06.730673441 +0000 UTC m=+309.674179671" Aug 04 22:26:07.334382 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.334351 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-58754c9c68-m54z8"] Aug 04 22:26:07.337342 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.337326 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.339457 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.339434 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Aug 04 22:26:07.339457 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.339447 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xmslk\"" Aug 04 22:26:07.340280 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.340253 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Aug 04 22:26:07.340378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.340296 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Aug 04 22:26:07.340378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.340300 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Aug 04 22:26:07.340378 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.340299 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-19der907s0n4d\"" Aug 04 22:26:07.345193 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.345174 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58754c9c68-m54z8"] Aug 04 22:26:07.496112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496083 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496114 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-client-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496136 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-metrics-server-audit-profiles\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496155 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/14c63d5d-571e-4485-beed-da671a98066c-audit-log\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496215 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqbzv\" (UniqueName: \"kubernetes.io/projected/14c63d5d-571e-4485-beed-da671a98066c-kube-api-access-jqbzv\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496257 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-client-certs\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.496294 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.496279 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-tls\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597345 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597269 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-client-certs\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597345 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597301 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-tls\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597345 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597340 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597364 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-client-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597392 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-metrics-server-audit-profiles\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597420 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/14c63d5d-571e-4485-beed-da671a98066c-audit-log\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597598 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597459 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqbzv\" (UniqueName: \"kubernetes.io/projected/14c63d5d-571e-4485-beed-da671a98066c-kube-api-access-jqbzv\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.597861 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.597836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/14c63d5d-571e-4485-beed-da671a98066c-audit-log\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.598154 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.598129 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.598603 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.598562 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/14c63d5d-571e-4485-beed-da671a98066c-metrics-server-audit-profiles\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.600159 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.600134 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-tls\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.600261 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.600247 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-secret-metrics-server-client-certs\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.600513 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.600494 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/14c63d5d-571e-4485-beed-da671a98066c-client-ca-bundle\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.605806 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.605784 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqbzv\" (UniqueName: \"kubernetes.io/projected/14c63d5d-571e-4485-beed-da671a98066c-kube-api-access-jqbzv\") pod \"metrics-server-58754c9c68-m54z8\" (UID: \"14c63d5d-571e-4485-beed-da671a98066c\") " pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.646972 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.646948 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:07.783447 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:07.783419 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-58754c9c68-m54z8"] Aug 04 22:26:07.786365 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:26:07.786338 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14c63d5d_571e_4485_beed_da671a98066c.slice/crio-329bbd705d123c5caf05771cef22eb4175b2c8172458c45efff48c2618314c62 WatchSource:0}: Error finding container 329bbd705d123c5caf05771cef22eb4175b2c8172458c45efff48c2618314c62: Status 404 returned error can't find the container with id 329bbd705d123c5caf05771cef22eb4175b2c8172458c45efff48c2618314c62 Aug 04 22:26:08.716967 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:08.716931 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" event={"ID":"14c63d5d-571e-4485-beed-da671a98066c","Type":"ContainerStarted","Data":"329bbd705d123c5caf05771cef22eb4175b2c8172458c45efff48c2618314c62"} Aug 04 22:26:09.657872 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:09.657803 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dcbff56cb-7jlgb" Aug 04 22:26:09.722479 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:09.722411 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" event={"ID":"14c63d5d-571e-4485-beed-da671a98066c","Type":"ContainerStarted","Data":"9a9e3980b0c710e532809f9f557920781374b131e39937d2300f945479a1a786"} Aug 04 22:26:09.745870 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:09.745821 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" podStartSLOduration=1.151291731 podStartE2EDuration="2.745804683s" totalImagesPullingTime="1.594512952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:26:09.74479551 +0000 UTC m=+312.688301745" watchObservedRunningTime="2026-08-04 22:26:09.745804683 +0000 UTC m=+312.689310914" Aug 04 22:26:27.647389 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:27.647354 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:27.647948 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:27.647434 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:33.339068 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:33.339006 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" podUID="4e19d1da-1437-4d92-9559-f6493cc65580" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 22:26:43.338860 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:43.338812 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" podUID="4e19d1da-1437-4d92-9559-f6493cc65580" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 22:26:45.186459 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:45.186430 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58754c9c68-m54z8_14c63d5d-571e-4485-beed-da671a98066c/metrics-server/0.log" Aug 04 22:26:45.585814 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:45.585787 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/init-textfile/0.log" Aug 04 22:26:45.787276 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:45.787249 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/node-exporter/0.log" Aug 04 22:26:45.987300 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:45.987274 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/kube-rbac-proxy/0.log" Aug 04 22:26:47.652459 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:47.652430 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:47.656159 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:47.656131 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-58754c9c68-m54z8" Aug 04 22:26:53.339923 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.339831 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" podUID="4e19d1da-1437-4d92-9559-f6493cc65580" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Aug 04 22:26:53.339923 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.339910 2571 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" Aug 04 22:26:53.340540 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.340504 2571 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"7232fe9ae564537e47bb980049abf5294a9daecc7dd759fb18d7d5612732a529"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" containerMessage="Container service-proxy failed liveness probe, will be restarted" Aug 04 22:26:53.340612 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.340568 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" podUID="4e19d1da-1437-4d92-9559-f6493cc65580" containerName="service-proxy" containerID="cri-o://7232fe9ae564537e47bb980049abf5294a9daecc7dd759fb18d7d5612732a529" gracePeriod=30 Aug 04 22:26:53.586707 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.586681 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wc957_efda0d95-7b75-435c-a1a9-e0eadf43d989/serve-healthcheck-canary/0.log" Aug 04 22:26:53.858339 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.858297 2571 generic.go:356] "Generic (PLEG): container finished" podID="4e19d1da-1437-4d92-9559-f6493cc65580" containerID="7232fe9ae564537e47bb980049abf5294a9daecc7dd759fb18d7d5612732a529" exitCode=2 Aug 04 22:26:53.858519 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.858385 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerDied","Data":"7232fe9ae564537e47bb980049abf5294a9daecc7dd759fb18d7d5612732a529"} Aug 04 22:26:53.858519 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:26:53.858423 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-559d964d76-h7b7v" event={"ID":"4e19d1da-1437-4d92-9559-f6493cc65580","Type":"ContainerStarted","Data":"43d954d869f79707d860239ad9391c4785a0e992d97b44e930cdf5f91afd3171"} Aug 04 22:27:10.449065 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.449005 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:27:10.451199 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.451179 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0dd7f79-567e-4af2-9787-647a72152f75-metrics-certs\") pod \"network-metrics-daemon-9gzwl\" (UID: \"f0dd7f79-567e-4af2-9787-647a72152f75\") " pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:27:10.499222 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.499201 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5596r\"" Aug 04 22:27:10.507013 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.506995 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9gzwl" Aug 04 22:27:10.626789 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.626759 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9gzwl"] Aug 04 22:27:10.630266 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:27:10.630239 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0dd7f79_567e_4af2_9787_647a72152f75.slice/crio-c35c52a4bc00eb40d8bc32aa8f94f27db6ca4191ba4c395253304f88fa1fcc6f WatchSource:0}: Error finding container c35c52a4bc00eb40d8bc32aa8f94f27db6ca4191ba4c395253304f88fa1fcc6f: Status 404 returned error can't find the container with id c35c52a4bc00eb40d8bc32aa8f94f27db6ca4191ba4c395253304f88fa1fcc6f Aug 04 22:27:10.911932 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:10.911899 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gzwl" event={"ID":"f0dd7f79-567e-4af2-9787-647a72152f75","Type":"ContainerStarted","Data":"c35c52a4bc00eb40d8bc32aa8f94f27db6ca4191ba4c395253304f88fa1fcc6f"} Aug 04 22:27:11.916967 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:11.916886 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gzwl" event={"ID":"f0dd7f79-567e-4af2-9787-647a72152f75","Type":"ContainerStarted","Data":"773157751abc222b4ea10fbf77e1724179ddcc68cf9eb342f35fa26843ec978e"} Aug 04 22:27:11.916967 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:11.916940 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9gzwl" event={"ID":"f0dd7f79-567e-4af2-9787-647a72152f75","Type":"ContainerStarted","Data":"3c758c9b898b1e9367b32f1d57b6f8974020c0dafaa81a980107351954c1ef13"} Aug 04 22:27:11.932503 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:27:11.932459 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9gzwl" podStartSLOduration=372.991966819 podStartE2EDuration="6m13.9324448s" totalImagesPullingTime="940.477981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:27:11.931846971 +0000 UTC m=+374.875353202" watchObservedRunningTime="2026-08-04 22:27:11.9324448 +0000 UTC m=+374.875951030" Aug 04 22:29:08.051617 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.051589 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659"] Aug 04 22:29:08.054653 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.054636 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.057334 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.057296 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 22:29:08.057472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.057343 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-mqf97\"" Aug 04 22:29:08.057472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.057365 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 22:29:08.057472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.057385 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 22:29:08.057472 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.057345 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 22:29:08.075261 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.075241 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659"] Aug 04 22:29:08.111182 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.111159 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.111274 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.111195 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.111274 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.111221 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll87j\" (UniqueName: \"kubernetes.io/projected/a36948aa-f972-4f29-9a76-bad94800429a-kube-api-access-ll87j\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.111344 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.111270 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-manifests\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.111344 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.111323 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-tmp\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212147 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212238 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212156 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212238 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll87j\" (UniqueName: \"kubernetes.io/projected/a36948aa-f972-4f29-9a76-bad94800429a-kube-api-access-ll87j\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212238 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212194 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-manifests\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212238 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212225 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-tmp\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.212644 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.212612 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-manifests\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.214439 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.214412 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a36948aa-f972-4f29-9a76-bad94800429a-tmp\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.214533 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.214517 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.214613 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.214595 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a36948aa-f972-4f29-9a76-bad94800429a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.219604 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.219578 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll87j\" (UniqueName: \"kubernetes.io/projected/a36948aa-f972-4f29-9a76-bad94800429a-kube-api-access-ll87j\") pod \"opendatahub-operator-controller-manager-7f6cdbbfdb-s2659\" (UID: \"a36948aa-f972-4f29-9a76-bad94800429a\") " pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.364452 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.364385 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:08.495049 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:08.494212 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659"] Aug 04 22:29:08.498456 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:29:08.498417 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36948aa_f972_4f29_9a76_bad94800429a.slice/crio-ae261abc7870bd4b264e487b6c0b65b37b17951b82a4406d9592d66a6de4d72d WatchSource:0}: Error finding container ae261abc7870bd4b264e487b6c0b65b37b17951b82a4406d9592d66a6de4d72d: Status 404 returned error can't find the container with id ae261abc7870bd4b264e487b6c0b65b37b17951b82a4406d9592d66a6de4d72d Aug 04 22:29:09.290613 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:09.290575 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" event={"ID":"a36948aa-f972-4f29-9a76-bad94800429a","Type":"ContainerStarted","Data":"ae261abc7870bd4b264e487b6c0b65b37b17951b82a4406d9592d66a6de4d72d"} Aug 04 22:29:12.303930 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:12.303891 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" event={"ID":"a36948aa-f972-4f29-9a76-bad94800429a","Type":"ContainerStarted","Data":"fa294a0c07064539e0a6b60c32c50f3bc3ec2a785e45457779928fd49070ba8e"} Aug 04 22:29:13.308775 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:13.308743 2571 generic.go:356] "Generic (PLEG): container finished" podID="a36948aa-f972-4f29-9a76-bad94800429a" containerID="fa294a0c07064539e0a6b60c32c50f3bc3ec2a785e45457779928fd49070ba8e" exitCode=0 Aug 04 22:29:13.309182 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:13.308819 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" event={"ID":"a36948aa-f972-4f29-9a76-bad94800429a","Type":"ContainerDied","Data":"fa294a0c07064539e0a6b60c32c50f3bc3ec2a785e45457779928fd49070ba8e"} Aug 04 22:29:14.314083 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:14.314027 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" event={"ID":"a36948aa-f972-4f29-9a76-bad94800429a","Type":"ContainerStarted","Data":"91953c6dfe3a2cbd6812b48e441230a6de41d105b78f86b5218b1fdd76781d8f"} Aug 04 22:29:14.314479 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:14.314187 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:14.335334 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:14.335291 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" podStartSLOduration=2.09626648 podStartE2EDuration="6.335275825s" totalImagesPullingTime="3.240922192s" totalInitContainerRuntime="998.087153ms" isStatefulPod=false podCreationTimestamp="2026-08-04 22:29:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:29:14.333222204 +0000 UTC m=+497.276728435" watchObservedRunningTime="2026-08-04 22:29:14.335275825 +0000 UTC m=+497.278782095" Aug 04 22:29:25.319728 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.319701 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f6cdbbfdb-s2659" Aug 04 22:29:25.465368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.465335 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg"] Aug 04 22:29:25.471642 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.471617 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.473951 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.473917 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Aug 04 22:29:25.474108 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.474063 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Aug 04 22:29:25.474186 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.474111 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Aug 04 22:29:25.474186 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.474170 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Aug 04 22:29:25.474342 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.474251 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Aug 04 22:29:25.474398 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.474357 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-27jgz\"" Aug 04 22:29:25.476245 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.476225 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg"] Aug 04 22:29:25.542819 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.542795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.542989 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.542836 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79b5d157-61d0-4351-a561-3dfcb01fe7f8-manager-config\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.542989 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.542865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-metrics-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.542989 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.542890 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nsb5\" (UniqueName: \"kubernetes.io/projected/79b5d157-61d0-4351-a561-3dfcb01fe7f8-kube-api-access-9nsb5\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.644273 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.644190 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.644273 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.644237 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79b5d157-61d0-4351-a561-3dfcb01fe7f8-manager-config\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.644273 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.644258 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-metrics-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.644273 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.644275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nsb5\" (UniqueName: \"kubernetes.io/projected/79b5d157-61d0-4351-a561-3dfcb01fe7f8-kube-api-access-9nsb5\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.644899 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.644879 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/79b5d157-61d0-4351-a561-3dfcb01fe7f8-manager-config\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.646688 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.646670 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.646807 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.646787 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/79b5d157-61d0-4351-a561-3dfcb01fe7f8-metrics-cert\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.651243 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.651223 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nsb5\" (UniqueName: \"kubernetes.io/projected/79b5d157-61d0-4351-a561-3dfcb01fe7f8-kube-api-access-9nsb5\") pod \"lws-controller-manager-76f49b9fdf-wbssg\" (UID: \"79b5d157-61d0-4351-a561-3dfcb01fe7f8\") " pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.782252 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.782215 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:25.903929 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:25.903853 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg"] Aug 04 22:29:25.906800 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:29:25.906776 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b5d157_61d0_4351_a561_3dfcb01fe7f8.slice/crio-e7a62f398aeabef696d5a150335c6df29e85c476e937f38f41ff8451a8ff9aec WatchSource:0}: Error finding container e7a62f398aeabef696d5a150335c6df29e85c476e937f38f41ff8451a8ff9aec: Status 404 returned error can't find the container with id e7a62f398aeabef696d5a150335c6df29e85c476e937f38f41ff8451a8ff9aec Aug 04 22:29:26.358284 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:26.358248 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" event={"ID":"79b5d157-61d0-4351-a561-3dfcb01fe7f8","Type":"ContainerStarted","Data":"e7a62f398aeabef696d5a150335c6df29e85c476e937f38f41ff8451a8ff9aec"} Aug 04 22:29:29.370506 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:29.370468 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" event={"ID":"79b5d157-61d0-4351-a561-3dfcb01fe7f8","Type":"ContainerStarted","Data":"7c39b2cabd885d9be12fc550a2bf307af2d9a1560b907f12a469fbb946644c7f"} Aug 04 22:29:29.370895 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:29.370610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:29.390533 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:29.390488 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" podStartSLOduration=1.6708344419999999 podStartE2EDuration="4.39047254s" totalImagesPullingTime="2.719638098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:29:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:29:29.390278107 +0000 UTC m=+512.333784341" watchObservedRunningTime="2026-08-04 22:29:29.39047254 +0000 UTC m=+512.333978772" Aug 04 22:29:35.472289 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.472256 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v"] Aug 04 22:29:35.475258 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.475243 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.477442 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477423 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 22:29:35.477564 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477451 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 22:29:35.477564 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477476 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dwq9r\"" Aug 04 22:29:35.477712 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477691 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 22:29:35.477836 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477819 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 22:29:35.477897 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477883 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 22:29:35.477961 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.477919 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 22:29:35.487857 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.487839 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v"] Aug 04 22:29:35.614801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.614801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614771 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86020cde-3513-4b1b-b491-b69292182c66-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.614801 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614788 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.615001 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614805 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.615001 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfrnm\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-kube-api-access-cfrnm\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.615001 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614911 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.615001 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.614950 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716072 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716025 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716179 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716093 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86020cde-3513-4b1b-b491-b69292182c66-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716179 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716115 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716179 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716140 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716179 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716167 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfrnm\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-kube-api-access-cfrnm\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716199 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716368 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716231 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.716844 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.716815 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.718655 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.718634 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.718655 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.718648 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.718791 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.718632 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86020cde-3513-4b1b-b491-b69292182c66-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.718791 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.718667 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86020cde-3513-4b1b-b491-b69292182c66-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.724204 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.724183 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfrnm\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-kube-api-access-cfrnm\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.724445 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.724427 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86020cde-3513-4b1b-b491-b69292182c66-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-ppc6v\" (UID: \"86020cde-3513-4b1b-b491-b69292182c66\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.784566 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.784543 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:35.923421 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:35.921252 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v"] Aug 04 22:29:36.393105 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:36.393067 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" event={"ID":"86020cde-3513-4b1b-b491-b69292182c66","Type":"ContainerStarted","Data":"2d37dcefeb43fd837e28893b0b8ec66537738cc188b5850b57f7b347365a9fda"} Aug 04 22:29:38.293841 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:38.293806 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:29:38.294152 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:38.293872 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:29:38.401646 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:38.401613 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" event={"ID":"86020cde-3513-4b1b-b491-b69292182c66","Type":"ContainerStarted","Data":"a485582268e2b24306e8118d8293fcaca0ba361dee124697bf5405b37a0a223c"} Aug 04 22:29:38.401790 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:38.401768 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:29:39.407486 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:39.407454 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-ppc6v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 22:29:39.407916 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:39.407501 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" podUID="86020cde-3513-4b1b-b491-b69292182c66" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 22:29:40.375517 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:40.375487 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76f49b9fdf-wbssg" Aug 04 22:29:40.392153 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:40.392103 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" podStartSLOduration=3.025420444 podStartE2EDuration="5.392087336s" totalImagesPullingTime="2.366666892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:29:38.434150928 +0000 UTC m=+521.377657168" watchObservedRunningTime="2026-08-04 22:29:40.392087336 +0000 UTC m=+523.335593566" Aug 04 22:29:42.407523 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:29:42.407491 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-ppc6v" Aug 04 22:31:37.601209 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.601183 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9"] Aug 04 22:31:37.604236 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.604214 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.606449 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.606429 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-62hmx\"" Aug 04 22:31:37.613068 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.613027 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9"] Aug 04 22:31:37.685718 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685689 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/312936f4-7905-4e61-bf78-46d9963e2b66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685723 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685739 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685764 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685857 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685907 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.685966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685940 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/312936f4-7905-4e61-bf78-46d9963e2b66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.686112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.685967 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzhr2\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-kube-api-access-vzhr2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.686112 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.686055 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787011 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.786983 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/312936f4-7905-4e61-bf78-46d9963e2b66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787015 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787030 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787063 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787086 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787108 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787124 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/312936f4-7905-4e61-bf78-46d9963e2b66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787140 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787141 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzhr2\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-kube-api-access-vzhr2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787485 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787163 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787485 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787467 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787594 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787491 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787649 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787614 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787730 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787709 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.787777 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.787719 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/312936f4-7905-4e61-bf78-46d9963e2b66-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.789462 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.789441 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/312936f4-7905-4e61-bf78-46d9963e2b66-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.789695 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.789674 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/312936f4-7905-4e61-bf78-46d9963e2b66-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.795459 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.795275 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.795459 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.795347 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzhr2\" (UniqueName: \"kubernetes.io/projected/312936f4-7905-4e61-bf78-46d9963e2b66-kube-api-access-vzhr2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5cmq9\" (UID: \"312936f4-7905-4e61-bf78-46d9963e2b66\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:37.915792 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:37.915741 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:38.041068 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:38.041015 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9"] Aug 04 22:31:38.043319 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:31:38.043295 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312936f4_7905_4e61_bf78_46d9963e2b66.slice/crio-9d3bca8be615a55f6417973aba71c939beb1e13093ddc3aac7f4fed5e061f0f3 WatchSource:0}: Error finding container 9d3bca8be615a55f6417973aba71c939beb1e13093ddc3aac7f4fed5e061f0f3: Status 404 returned error can't find the container with id 9d3bca8be615a55f6417973aba71c939beb1e13093ddc3aac7f4fed5e061f0f3 Aug 04 22:31:38.045356 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:38.045341 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 22:31:38.806529 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:38.806496 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" event={"ID":"312936f4-7905-4e61-bf78-46d9963e2b66","Type":"ContainerStarted","Data":"9d3bca8be615a55f6417973aba71c939beb1e13093ddc3aac7f4fed5e061f0f3"} Aug 04 22:31:45.423986 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.423946 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:31:45.424243 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.424029 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:31:45.424243 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.424073 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:31:45.832163 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.832131 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" event={"ID":"312936f4-7905-4e61-bf78-46d9963e2b66","Type":"ContainerStarted","Data":"5068fea01696a0f63c16caeb68126600d0873f42a3105635dbff2378ea1efc6d"} Aug 04 22:31:45.854071 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.854004 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podStartSLOduration=1.475784827 podStartE2EDuration="8.853989286s" totalImagesPullingTime="7.378204459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:31:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:31:45.852370032 +0000 UTC m=+648.795876262" watchObservedRunningTime="2026-08-04 22:31:45.853989286 +0000 UTC m=+648.797495518" Aug 04 22:31:45.915872 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:45.915841 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:46.921934 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:46.921905 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:47.723950 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.723918 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:47.727045 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.727010 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:47.729305 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.729286 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 22:31:47.729974 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.729954 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 22:31:47.729974 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.729962 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-2f6x4\"" Aug 04 22:31:47.733760 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.733556 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:47.758289 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.758265 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhpl7\" (UniqueName: \"kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7\") pod \"authorino-b54df94b4-crrs6\" (UID: \"9cb682ba-c67a-49b5-a13d-cffd1b414fa4\") " pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:47.842736 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.842711 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:47.843642 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.843624 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:31:47.859309 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.859286 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhpl7\" (UniqueName: \"kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7\") pod \"authorino-b54df94b4-crrs6\" (UID: \"9cb682ba-c67a-49b5-a13d-cffd1b414fa4\") " pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:47.868311 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:47.868283 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhpl7\" (UniqueName: \"kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7\") pod \"authorino-b54df94b4-crrs6\" (UID: \"9cb682ba-c67a-49b5-a13d-cffd1b414fa4\") " pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:48.036588 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:48.036506 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:48.155108 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:48.155085 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:48.157605 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:31:48.157580 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cb682ba_c67a_49b5_a13d_cffd1b414fa4.slice/crio-e3e2b09c260a8fd2cd3570ff9681e110c36223274d9a83f291c5f78da8120ffd WatchSource:0}: Error finding container e3e2b09c260a8fd2cd3570ff9681e110c36223274d9a83f291c5f78da8120ffd: Status 404 returned error can't find the container with id e3e2b09c260a8fd2cd3570ff9681e110c36223274d9a83f291c5f78da8120ffd Aug 04 22:31:48.847493 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:48.847462 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-crrs6" event={"ID":"9cb682ba-c67a-49b5-a13d-cffd1b414fa4","Type":"ContainerStarted","Data":"e3e2b09c260a8fd2cd3570ff9681e110c36223274d9a83f291c5f78da8120ffd"} Aug 04 22:31:51.859967 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:51.859933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-crrs6" event={"ID":"9cb682ba-c67a-49b5-a13d-cffd1b414fa4","Type":"ContainerStarted","Data":"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b"} Aug 04 22:31:51.873849 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:51.873803 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-crrs6" podStartSLOduration=1.310209785 podStartE2EDuration="4.873789241s" totalImagesPullingTime="3.563579456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:31:51.873185118 +0000 UTC m=+654.816691356" watchObservedRunningTime="2026-08-04 22:31:51.873789241 +0000 UTC m=+654.817295501" Aug 04 22:31:52.632487 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:52.632459 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:53.866115 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:53.866078 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-crrs6" podUID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" containerName="authorino" containerID="cri-o://5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b" gracePeriod=30 Aug 04 22:31:54.107137 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.107113 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:54.214489 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.214459 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhpl7\" (UniqueName: \"kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7\") pod \"9cb682ba-c67a-49b5-a13d-cffd1b414fa4\" (UID: \"9cb682ba-c67a-49b5-a13d-cffd1b414fa4\") " Aug 04 22:31:54.216522 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.216502 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7" (OuterVolumeSpecName: "kube-api-access-hhpl7") pod "9cb682ba-c67a-49b5-a13d-cffd1b414fa4" (UID: "9cb682ba-c67a-49b5-a13d-cffd1b414fa4"). InnerVolumeSpecName "kube-api-access-hhpl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:31:54.314929 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.314899 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhpl7\" (UniqueName: \"kubernetes.io/projected/9cb682ba-c67a-49b5-a13d-cffd1b414fa4-kube-api-access-hhpl7\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:31:54.870865 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.870832 2571 generic.go:356] "Generic (PLEG): container finished" podID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" containerID="5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b" exitCode=0 Aug 04 22:31:54.871286 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.870873 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-crrs6" Aug 04 22:31:54.871286 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.870909 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-crrs6" event={"ID":"9cb682ba-c67a-49b5-a13d-cffd1b414fa4","Type":"ContainerDied","Data":"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b"} Aug 04 22:31:54.871286 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.870947 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-crrs6" event={"ID":"9cb682ba-c67a-49b5-a13d-cffd1b414fa4","Type":"ContainerDied","Data":"e3e2b09c260a8fd2cd3570ff9681e110c36223274d9a83f291c5f78da8120ffd"} Aug 04 22:31:54.871286 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.870980 2571 scope.go:117] "RemoveContainer" containerID="5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b" Aug 04 22:31:54.879447 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.879429 2571 scope.go:117] "RemoveContainer" containerID="5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b" Aug 04 22:31:54.879700 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:31:54.879679 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b\": container with ID starting with 5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b not found: ID does not exist" containerID="5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b" Aug 04 22:31:54.879757 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.879712 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b"} err="failed to get container status \"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b\": rpc error: code = NotFound desc = could not find container \"5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b\": container with ID starting with 5be7d3d7aa7d8f94e787c08899a51236f4c7acf59befa98b38257e35ffbc941b not found: ID does not exist" Aug 04 22:31:54.890614 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.890589 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:54.895974 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:54.895954 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-crrs6"] Aug 04 22:31:55.605197 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:31:55.601962 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" path="/var/lib/kubelet/pods/9cb682ba-c67a-49b5-a13d-cffd1b414fa4/volumes" Aug 04 22:32:03.456055 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.456001 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzm47"] Aug 04 22:32:03.456427 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.456378 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" containerName="authorino" Aug 04 22:32:03.456427 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.456395 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" containerName="authorino" Aug 04 22:32:03.456502 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.456461 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb682ba-c67a-49b5-a13d-cffd1b414fa4" containerName="authorino" Aug 04 22:32:03.460656 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.460635 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.463136 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.463115 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Aug 04 22:32:03.463245 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.463120 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Aug 04 22:32:03.463815 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.463798 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-l2r79\"" Aug 04 22:32:03.463886 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.463812 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Aug 04 22:32:03.471183 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.471163 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzm47"] Aug 04 22:32:03.584828 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.584797 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff32be7b-f411-430c-be28-69fd484fd046-data\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.584975 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.584846 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgqf2\" (UniqueName: \"kubernetes.io/projected/ff32be7b-f411-430c-be28-69fd484fd046-kube-api-access-zgqf2\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.686126 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.686096 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff32be7b-f411-430c-be28-69fd484fd046-data\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.686287 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.686143 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgqf2\" (UniqueName: \"kubernetes.io/projected/ff32be7b-f411-430c-be28-69fd484fd046-kube-api-access-zgqf2\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.686501 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.686482 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ff32be7b-f411-430c-be28-69fd484fd046-data\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.694738 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.694708 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgqf2\" (UniqueName: \"kubernetes.io/projected/ff32be7b-f411-430c-be28-69fd484fd046-kube-api-access-zgqf2\") pod \"postgres-69fb4594cf-hzm47\" (UID: \"ff32be7b-f411-430c-be28-69fd484fd046\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.771158 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.771091 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:03.892205 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.892183 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hzm47"] Aug 04 22:32:03.894339 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:32:03.894314 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff32be7b_f411_430c_be28_69fd484fd046.slice/crio-3f92110e4819f838b345369f2f9f5fd8b0b111cdb651c8d2c384c23bc7d17403 WatchSource:0}: Error finding container 3f92110e4819f838b345369f2f9f5fd8b0b111cdb651c8d2c384c23bc7d17403: Status 404 returned error can't find the container with id 3f92110e4819f838b345369f2f9f5fd8b0b111cdb651c8d2c384c23bc7d17403 Aug 04 22:32:03.903071 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:03.903028 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" event={"ID":"ff32be7b-f411-430c-be28-69fd484fd046","Type":"ContainerStarted","Data":"3f92110e4819f838b345369f2f9f5fd8b0b111cdb651c8d2c384c23bc7d17403"} Aug 04 22:32:08.926627 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:08.926597 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" event={"ID":"ff32be7b-f411-430c-be28-69fd484fd046","Type":"ContainerStarted","Data":"c6f942465aa1e33a21345142025ed70e21ee9ad1e112c1da84f9435f59463cdb"} Aug 04 22:32:08.926978 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:08.926706 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:32:08.943600 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:08.943556 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" podStartSLOduration=1.049942048 podStartE2EDuration="5.943541985s" totalImagesPullingTime="4.893599937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:32:08.941528347 +0000 UTC m=+671.885034580" watchObservedRunningTime="2026-08-04 22:32:08.943541985 +0000 UTC m=+671.887048217" Aug 04 22:32:14.958594 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:32:14.958561 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hzm47" Aug 04 22:33:09.136777 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.136690 2571 generic.go:356] "Generic (PLEG): container finished" podID="312936f4-7905-4e61-bf78-46d9963e2b66" containerID="5068fea01696a0f63c16caeb68126600d0873f42a3105635dbff2378ea1efc6d" exitCode=0 Aug 04 22:33:09.136777 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.136735 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" event={"ID":"312936f4-7905-4e61-bf78-46d9963e2b66","Type":"ContainerDied","Data":"5068fea01696a0f63c16caeb68126600d0873f42a3105635dbff2378ea1efc6d"} Aug 04 22:33:09.137220 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.137072 2571 scope.go:117] "RemoveContainer" containerID="5068fea01696a0f63c16caeb68126600d0873f42a3105635dbff2378ea1efc6d" Aug 04 22:33:09.138075 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.138028 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:33:09.138144 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.138124 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:33:09.138180 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:09.138156 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:33:10.157747 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:10.157708 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" event={"ID":"312936f4-7905-4e61-bf78-46d9963e2b66","Type":"ContainerStarted","Data":"e489495966a666f26edada067e1f96645f49f4c777002000dcfcc12b77fa7ba8"} Aug 04 22:33:10.915866 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:10.915827 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:33:10.916190 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:10.916166 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5cmq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body= Aug 04 22:33:10.916246 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:10.916228 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podUID="312936f4-7905-4e61-bf78-46d9963e2b66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" Aug 04 22:33:11.916214 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:11.916179 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5cmq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body= Aug 04 22:33:11.916591 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:11.916232 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podUID="312936f4-7905-4e61-bf78-46d9963e2b66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" Aug 04 22:33:12.916497 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:12.916463 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5cmq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body= Aug 04 22:33:12.916865 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:12.916526 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podUID="312936f4-7905-4e61-bf78-46d9963e2b66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" Aug 04 22:33:13.916869 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:13.916834 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5cmq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body= Aug 04 22:33:13.917280 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:13.916898 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podUID="312936f4-7905-4e61-bf78-46d9963e2b66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" Aug 04 22:33:14.917155 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:14.917121 2571 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5cmq9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" start-of-body= Aug 04 22:33:14.917519 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:14.917174 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" podUID="312936f4-7905-4e61-bf78-46d9963e2b66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.16:15021/healthz/ready\": dial tcp 10.133.0.16:15021: connect: connection refused" Aug 04 22:33:15.921781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:15.921752 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:33:15.922190 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:15.922012 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:33:15.922855 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:15.922836 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5cmq9" Aug 04 22:33:43.192841 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.192807 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt"] Aug 04 22:33:43.196408 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.196383 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.199486 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.199466 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Aug 04 22:33:43.199603 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.199511 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8mcsf\"" Aug 04 22:33:43.199603 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.199558 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 22:33:43.199728 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.199617 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 22:33:43.205539 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.205517 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt"] Aug 04 22:33:43.342701 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342676 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.342868 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342714 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.342868 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342747 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.342868 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342818 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.342868 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342839 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.343016 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.342881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99z99\" (UniqueName: \"kubernetes.io/projected/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-kube-api-access-99z99\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443666 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443586 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443666 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443622 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443666 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443647 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443912 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443766 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443912 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443795 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.443912 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.443830 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99z99\" (UniqueName: \"kubernetes.io/projected/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-kube-api-access-99z99\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.444212 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.444187 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.444212 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.444216 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.444372 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.444241 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.445881 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.445856 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.446018 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.446002 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.451146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.451123 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99z99\" (UniqueName: \"kubernetes.io/projected/22b1c7cb-4142-44cf-86b9-87d7c920b6a0-kube-api-access-99z99\") pod \"e2e-distinct-simulated-kserve-59975d69f9-g4ckt\" (UID: \"22b1c7cb-4142-44cf-86b9-87d7c920b6a0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.508104 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.508080 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:43.633201 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:43.633092 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt"] Aug 04 22:33:43.635715 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:33:43.635687 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b1c7cb_4142_44cf_86b9_87d7c920b6a0.slice/crio-b9aa6898403d7b71d27f7fbf694e665cfaac51b01ec4e2df93da71e315138465 WatchSource:0}: Error finding container b9aa6898403d7b71d27f7fbf694e665cfaac51b01ec4e2df93da71e315138465: Status 404 returned error can't find the container with id b9aa6898403d7b71d27f7fbf694e665cfaac51b01ec4e2df93da71e315138465 Aug 04 22:33:44.283492 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:44.283419 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" event={"ID":"22b1c7cb-4142-44cf-86b9-87d7c920b6a0","Type":"ContainerStarted","Data":"b9aa6898403d7b71d27f7fbf694e665cfaac51b01ec4e2df93da71e315138465"} Aug 04 22:33:45.288186 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:45.288147 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" event={"ID":"22b1c7cb-4142-44cf-86b9-87d7c920b6a0","Type":"ContainerStarted","Data":"e54f5bfef156ff48477d80b98007dadb64df395e1a06d75401cb88711bd3ca9c"} Aug 04 22:33:45.304287 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:45.304234 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" podStartSLOduration=1.340365805 podStartE2EDuration="2.304218461s" totalImagesPullingTime="963.852656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:33:45.302974416 +0000 UTC m=+768.246480646" watchObservedRunningTime="2026-08-04 22:33:45.304218461 +0000 UTC m=+768.247724692" Aug 04 22:33:53.509183 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:53.509153 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:53.509602 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:53.509209 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:53.521564 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:53.521539 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:33:54.329742 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:33:54.329706 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-g4ckt" Aug 04 22:34:31.505833 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.505747 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj"] Aug 04 22:34:31.508646 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.508630 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.510827 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.510806 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Aug 04 22:34:31.516998 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.516976 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj"] Aug 04 22:34:31.584855 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.584822 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9snc6\" (UniqueName: \"kubernetes.io/projected/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-kube-api-access-9snc6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.585005 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.584865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.585005 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.584929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.585005 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.584971 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.585191 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.585085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.585191 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.585149 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686433 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686404 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686433 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686687 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686550 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686687 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686576 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686687 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686629 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9snc6\" (UniqueName: \"kubernetes.io/projected/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-kube-api-access-9snc6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686848 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686692 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686903 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686848 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.686903 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.686883 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.687081 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.687030 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.688941 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.688922 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.689247 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.689229 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.694386 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.694364 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9snc6\" (UniqueName: \"kubernetes.io/projected/6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f-kube-api-access-9snc6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj\" (UID: \"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.819085 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.818990 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:31.940069 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:31.940028 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj"] Aug 04 22:34:31.943729 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:34:31.943701 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b787a9a_ee4e_4173_9e1f_c1f867c9fb5f.slice/crio-f573444cba7583e274165c270bb9f13e8e18931ffd83753bdd47347055a8f9e6 WatchSource:0}: Error finding container f573444cba7583e274165c270bb9f13e8e18931ffd83753bdd47347055a8f9e6: Status 404 returned error can't find the container with id f573444cba7583e274165c270bb9f13e8e18931ffd83753bdd47347055a8f9e6 Aug 04 22:34:32.461087 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:32.461055 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" event={"ID":"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f","Type":"ContainerStarted","Data":"02c5e32db9c7926e199884fe789b5d633ff5ad3ae996d671e05ea221ce7b9284"} Aug 04 22:34:32.461263 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:32.461117 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" event={"ID":"6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f","Type":"ContainerStarted","Data":"f573444cba7583e274165c270bb9f13e8e18931ffd83753bdd47347055a8f9e6"} Aug 04 22:34:32.478111 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:32.478062 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" podStartSLOduration=1.3042376949999999 podStartE2EDuration="1.478032421s" totalImagesPullingTime="173.794726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:34:32.476167949 +0000 UTC m=+815.419674181" watchObservedRunningTime="2026-08-04 22:34:32.478032421 +0000 UTC m=+815.421538652" Aug 04 22:34:41.819953 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:41.819912 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:41.820367 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:41.819983 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:41.832574 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:41.832546 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:42.511806 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:42.511772 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj" Aug 04 22:34:51.377376 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.377343 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs"] Aug 04 22:34:51.382982 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.382959 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.385376 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.385355 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Aug 04 22:34:51.390751 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.390727 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs"] Aug 04 22:34:51.542200 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542175 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.542343 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542214 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkwsv\" (UniqueName: \"kubernetes.io/projected/aabc8086-1c11-4c3b-a675-916c15f36ef4-kube-api-access-gkwsv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.542343 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542247 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.542343 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aabc8086-1c11-4c3b-a675-916c15f36ef4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.542446 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542359 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.542446 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.542380 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643696 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643619 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643696 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643675 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aabc8086-1c11-4c3b-a675-916c15f36ef4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643696 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643693 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643712 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643734 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.643966 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.643760 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkwsv\" (UniqueName: \"kubernetes.io/projected/aabc8086-1c11-4c3b-a675-916c15f36ef4-kube-api-access-gkwsv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.644172 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.644148 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.644220 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.644204 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.644255 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.644225 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.645845 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.645824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aabc8086-1c11-4c3b-a675-916c15f36ef4-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.646130 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.646109 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aabc8086-1c11-4c3b-a675-916c15f36ef4-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.651433 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.651411 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkwsv\" (UniqueName: \"kubernetes.io/projected/aabc8086-1c11-4c3b-a675-916c15f36ef4-kube-api-access-gkwsv\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-j89hs\" (UID: \"aabc8086-1c11-4c3b-a675-916c15f36ef4\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.692474 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.692454 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:34:51.813721 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:51.813699 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs"] Aug 04 22:34:51.816071 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:34:51.816030 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaabc8086_1c11_4c3b_a675_916c15f36ef4.slice/crio-de03568fda3d7973cd88bac0bf3d38ff51186cccd8082c6955f8d190eb0b37e1 WatchSource:0}: Error finding container de03568fda3d7973cd88bac0bf3d38ff51186cccd8082c6955f8d190eb0b37e1: Status 404 returned error can't find the container with id de03568fda3d7973cd88bac0bf3d38ff51186cccd8082c6955f8d190eb0b37e1 Aug 04 22:34:52.540580 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:52.540550 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" event={"ID":"aabc8086-1c11-4c3b-a675-916c15f36ef4","Type":"ContainerStarted","Data":"c2d8168ba81d1df6f6cba7465a08a9f19a977d2f5565f88aff6a0b0903da2add"} Aug 04 22:34:52.540928 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:52.540595 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" event={"ID":"aabc8086-1c11-4c3b-a675-916c15f36ef4","Type":"ContainerStarted","Data":"de03568fda3d7973cd88bac0bf3d38ff51186cccd8082c6955f8d190eb0b37e1"} Aug 04 22:34:52.558871 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:34:52.558830 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" podStartSLOduration=1.397773167 podStartE2EDuration="1.558817198s" totalImagesPullingTime="161.044031ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:34:52.558118405 +0000 UTC m=+835.501624636" watchObservedRunningTime="2026-08-04 22:34:52.558817198 +0000 UTC m=+835.502323428" Aug 04 22:35:01.692650 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:35:01.692617 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:35:01.693069 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:35:01.692686 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:35:01.704991 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:35:01.704969 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:35:02.596158 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:35:02.596125 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-j89hs" Aug 04 22:53:44.452824 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.452790 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:53:44.455867 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.455852 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.459845 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.459823 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-63f65db9-gw-openshift-default-dockercfg-rjt7v\"" Aug 04 22:53:44.493935 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.493900 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:53:44.594853 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.594800 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595065 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.594996 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595142 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595061 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595298 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595109 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595364 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595349 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595408 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595396 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595448 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595415 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595484 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595466 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.595521 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.595503 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47ncn\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696752 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696703 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47ncn\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696752 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696762 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696781 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696800 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696827 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696910 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.696949 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696937 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697258 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.696963 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.697301 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697404 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.697381 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697483 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.697460 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697522 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.697486 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.697689 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.697667 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.699206 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.699183 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.699475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.699459 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.705105 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.705055 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.705246 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.705229 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47ncn\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn\") pod \"e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.766046 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.766010 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:44.895671 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.895644 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:53:44.898249 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:53:44.898224 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49168781_2c16_4865_a675_7460cfbd0e30.slice/crio-28b516bed4b22c6daed17062dc7bb41533155a84b8a2465bdc3442735f423d45 WatchSource:0}: Error finding container 28b516bed4b22c6daed17062dc7bb41533155a84b8a2465bdc3442735f423d45: Status 404 returned error can't find the container with id 28b516bed4b22c6daed17062dc7bb41533155a84b8a2465bdc3442735f423d45 Aug 04 22:53:44.900481 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.900449 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:53:44.900569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.900511 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:53:44.900569 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:44.900540 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:53:45.004013 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:45.003933 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerStarted","Data":"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5"} Aug 04 22:53:45.004013 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:45.003991 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerStarted","Data":"28b516bed4b22c6daed17062dc7bb41533155a84b8a2465bdc3442735f423d45"} Aug 04 22:53:45.024282 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:45.024229 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podStartSLOduration=1.024210084 podStartE2EDuration="1.024210084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:53:45.023176017 +0000 UTC m=+1967.966682249" watchObservedRunningTime="2026-08-04 22:53:45.024210084 +0000 UTC m=+1967.967716308" Aug 04 22:53:45.766776 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:45.766741 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:45.771460 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:45.771436 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:46.009434 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:46.009397 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:53:46.010376 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:53:46.010360 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:00.068665 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.068631 2571 generic.go:356] "Generic (PLEG): container finished" podID="49168781-2c16-4865-a675-7460cfbd0e30" containerID="90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5" exitCode=0 Aug 04 22:54:00.069172 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.068709 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerDied","Data":"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5"} Aug 04 22:54:00.069172 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.069159 2571 scope.go:117] "RemoveContainer" containerID="90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5" Aug 04 22:54:00.070154 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.070126 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:54:00.070254 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.070190 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:54:00.070254 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:00.070222 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:54:01.009866 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:01.009824 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:01.074895 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:01.074862 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerStarted","Data":"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8"} Aug 04 22:54:01.767097 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:01.767056 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:01.768180 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:01.768149 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body= Aug 04 22:54:01.768309 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:01.768203 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" Aug 04 22:54:02.767267 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:02.767227 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body= Aug 04 22:54:02.767657 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:02.767309 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" Aug 04 22:54:03.767078 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:03.766539 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body= Aug 04 22:54:03.767078 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:03.766607 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" Aug 04 22:54:04.766430 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:04.766391 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body= Aug 04 22:54:04.767100 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:04.766993 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" Aug 04 22:54:05.766498 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:05.766457 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" start-of-body= Aug 04 22:54:05.767124 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:05.767078 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.22:15021/healthz/ready\": dial tcp 10.133.0.22:15021: connect: connection refused" Aug 04 22:54:06.770564 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:06.770529 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:06.771006 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:06.770824 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:06.771590 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:06.771574 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:17.786528 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:17.786493 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:54:17.787025 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:17.786691 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" containerID="cri-o://ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8" gracePeriod=30 Aug 04 22:54:21.772866 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:21.772826 2571 patch_prober.go:28] interesting pod/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 22:54:21.773259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:21.772888 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 22:54:23.026400 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.026377 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:23.120091 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120006 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120091 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120068 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120091 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120089 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120133 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120148 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120166 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120207 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120250 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47ncn\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120287 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket\") pod \"49168781-2c16-4865-a675-7460cfbd0e30\" (UID: \"49168781-2c16-4865-a675-7460cfbd0e30\") " Aug 04 22:54:23.120333 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120289 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:54:23.120664 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120456 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data" (OuterVolumeSpecName: "istio-data") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:54:23.120664 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120464 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.120664 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120540 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:54:23.120664 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120558 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 22:54:23.120664 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.120645 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:54:23.122507 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.122469 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 22:54:23.122507 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.122491 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:54:23.122645 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.122575 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn" (OuterVolumeSpecName: "kube-api-access-47ncn") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "kube-api-access-47ncn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:54:23.122645 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.122581 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token" (OuterVolumeSpecName: "istio-token") pod "49168781-2c16-4865-a675-7460cfbd0e30" (UID: "49168781-2c16-4865-a675-7460cfbd0e30"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:54:23.169312 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.169285 2571 generic.go:356] "Generic (PLEG): container finished" podID="49168781-2c16-4865-a675-7460cfbd0e30" containerID="ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8" exitCode=0 Aug 04 22:54:23.169412 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.169329 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" Aug 04 22:54:23.169412 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.169365 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerDied","Data":"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8"} Aug 04 22:54:23.169494 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.169414 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf" event={"ID":"49168781-2c16-4865-a675-7460cfbd0e30","Type":"ContainerDied","Data":"28b516bed4b22c6daed17062dc7bb41533155a84b8a2465bdc3442735f423d45"} Aug 04 22:54:23.169494 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.169438 2571 scope.go:117] "RemoveContainer" containerID="ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8" Aug 04 22:54:23.177478 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.177462 2571 scope.go:117] "RemoveContainer" containerID="90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5" Aug 04 22:54:23.184391 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.184374 2571 scope.go:117] "RemoveContainer" containerID="ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8" Aug 04 22:54:23.184627 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:54:23.184609 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8\": container with ID starting with ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8 not found: ID does not exist" containerID="ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8" Aug 04 22:54:23.184699 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.184639 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8"} err="failed to get container status \"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8\": rpc error: code = NotFound desc = could not find container \"ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8\": container with ID starting with ce41cc3f07a02db82b3d541203f0293d7ab68a0518b5420dc026894d3a3a55b8 not found: ID does not exist" Aug 04 22:54:23.184699 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.184663 2571 scope.go:117] "RemoveContainer" containerID="90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5" Aug 04 22:54:23.184918 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:54:23.184900 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5\": container with ID starting with 90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5 not found: ID does not exist" containerID="90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5" Aug 04 22:54:23.184960 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.184923 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5"} err="failed to get container status \"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5\": rpc error: code = NotFound desc = could not find container \"90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5\": container with ID starting with 90eddd1e300913485f6fece97289abdab01d949e7598a26bb47b61b61f1606e5 not found: ID does not exist" Aug 04 22:54:23.189608 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.189581 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:54:23.193487 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.193467 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-63f65db9-gw-openshift-default-76967ffcf6-zdsdf"] Aug 04 22:54:23.221457 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221437 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221457 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221455 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221465 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/49168781-2c16-4865-a675-7460cfbd0e30-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221474 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221484 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47ncn\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-kube-api-access-47ncn\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221494 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/49168781-2c16-4865-a675-7460cfbd0e30-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221503 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/49168781-2c16-4865-a675-7460cfbd0e30-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.221558 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.221511 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/49168781-2c16-4865-a675-7460cfbd0e30-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:54:23.601082 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:54:23.601027 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49168781-2c16-4865-a675-7460cfbd0e30" path="/var/lib/kubelet/pods/49168781-2c16-4865-a675-7460cfbd0e30/volumes" Aug 04 22:55:04.768595 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.768558 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.768959 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.768977 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.769004 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.769013 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.769101 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.769160 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.769112 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="49168781-2c16-4865-a675-7460cfbd0e30" containerName="istio-proxy" Aug 04 22:55:04.770872 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.770852 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.773513 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.773488 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-8410bf38-openshift-default-dockercfg-5fdmk\"" Aug 04 22:55:04.785424 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.785402 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:04.953795 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953751 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.953993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953804 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.953993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953877 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcnp\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.953993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953933 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.953993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953963 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.953993 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.953981 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.954296 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.954013 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.954296 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.954028 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:04.954296 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:04.954066 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055030 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.054958 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055030 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.054991 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055030 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055007 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055054 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055078 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055102 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055136 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055196 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055259 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055232 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcnp\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055544 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055410 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055544 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055456 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055653 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055629 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055701 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055647 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.055869 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.055853 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.057765 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.057740 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.057972 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.057954 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.063061 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.063005 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.063146 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.063114 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcnp\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp\") pod \"e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.082982 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.082965 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:05.221822 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.221793 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:05.223529 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:55:05.223504 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf2f195_6889_4bbe_af7d_34de349c4c9f.slice/crio-b5b3e3a4b4f148214c8da34fe818894009ab5bde9fbc80df0a62089cbd234fee WatchSource:0}: Error finding container b5b3e3a4b4f148214c8da34fe818894009ab5bde9fbc80df0a62089cbd234fee: Status 404 returned error can't find the container with id b5b3e3a4b4f148214c8da34fe818894009ab5bde9fbc80df0a62089cbd234fee Aug 04 22:55:05.225660 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.225622 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:05.225745 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.225705 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:05.225786 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.225743 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:05.339475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.339391 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerStarted","Data":"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54"} Aug 04 22:55:05.339475 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.339462 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerStarted","Data":"b5b3e3a4b4f148214c8da34fe818894009ab5bde9fbc80df0a62089cbd234fee"} Aug 04 22:55:05.363840 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:05.363789 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podStartSLOduration=1.363773562 podStartE2EDuration="1.363773562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:55:05.362715807 +0000 UTC m=+2048.306222051" watchObservedRunningTime="2026-08-04 22:55:05.363773562 +0000 UTC m=+2048.307279793" Aug 04 22:55:06.083980 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:06.083943 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:06.088773 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:06.088751 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:06.343349 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:06.343266 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:06.344284 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:06.344260 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:20.401354 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.401317 2571 generic.go:356] "Generic (PLEG): container finished" podID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerID="8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54" exitCode=0 Aug 04 22:55:20.401747 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.401388 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerDied","Data":"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54"} Aug 04 22:55:20.401788 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.401748 2571 scope.go:117] "RemoveContainer" containerID="8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54" Aug 04 22:55:20.402855 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.402827 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:20.402959 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.402892 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:20.402959 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:20.402922 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:55:21.343821 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:21.343783 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:21.407262 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:21.407232 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerStarted","Data":"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f"} Aug 04 22:55:22.084030 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:22.083984 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:22.085158 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:22.085129 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Aug 04 22:55:22.085279 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:22.085182 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Aug 04 22:55:23.083730 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:23.083694 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Aug 04 22:55:23.084116 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:23.083757 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Aug 04 22:55:24.083879 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:24.083844 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Aug 04 22:55:24.084274 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:24.083912 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Aug 04 22:55:25.083649 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:25.083613 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Aug 04 22:55:25.083813 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:25.083682 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Aug 04 22:55:26.084106 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:26.084068 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" start-of-body= Aug 04 22:55:26.084483 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:26.084136 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.23:15021/healthz/ready\": dial tcp 10.133.0.23:15021: connect: connection refused" Aug 04 22:55:27.087926 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:27.087894 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:27.088329 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:27.088237 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:27.088781 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:27.088763 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:38.717288 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:38.717251 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:38.717726 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:38.717466 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" containerID="cri-o://fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f" gracePeriod=30 Aug 04 22:55:42.090235 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:42.090190 2571 patch_prober.go:28] interesting pod/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Aug 04 22:55:42.090657 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:42.090253 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 22:55:43.962120 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:43.962098 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:44.077643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077581 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077617 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077643 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077641 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077656 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077705 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077720 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077745 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwcnp\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077786 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077839 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data" (OuterVolumeSpecName: "istio-data") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077853 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token\") pod \"caf2f195-6889-4bbe-af7d-34de349c4c9f\" (UID: \"caf2f195-6889-4bbe-af7d-34de349c4c9f\") " Aug 04 22:55:44.077908 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077862 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.077920 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.078165 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.078186 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.078197 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.078230 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:55:44.078324 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.078249 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 22:55:44.079892 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.079866 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp" (OuterVolumeSpecName: "kube-api-access-gwcnp") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "kube-api-access-gwcnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:55:44.079892 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.079879 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:55:44.080286 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.080268 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token" (OuterVolumeSpecName: "istio-token") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:55:44.080364 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.080269 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "caf2f195-6889-4bbe-af7d-34de349c4c9f" (UID: "caf2f195-6889-4bbe-af7d-34de349c4c9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 22:55:44.178810 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178775 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.178810 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178808 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.178810 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178816 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf2f195-6889-4bbe-af7d-34de349c4c9f-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.178984 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178825 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf2f195-6889-4bbe-af7d-34de349c4c9f-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.178984 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178834 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwcnp\" (UniqueName: \"kubernetes.io/projected/caf2f195-6889-4bbe-af7d-34de349c4c9f-kube-api-access-gwcnp\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.178984 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.178843 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf2f195-6889-4bbe-af7d-34de349c4c9f-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:55:44.501655 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.501621 2571 generic.go:356] "Generic (PLEG): container finished" podID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerID="fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f" exitCode=0 Aug 04 22:55:44.501805 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.501668 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" Aug 04 22:55:44.501805 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.501711 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerDied","Data":"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f"} Aug 04 22:55:44.501805 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.501770 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc" event={"ID":"caf2f195-6889-4bbe-af7d-34de349c4c9f","Type":"ContainerDied","Data":"b5b3e3a4b4f148214c8da34fe818894009ab5bde9fbc80df0a62089cbd234fee"} Aug 04 22:55:44.501805 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.501799 2571 scope.go:117] "RemoveContainer" containerID="fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f" Aug 04 22:55:44.510818 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.510801 2571 scope.go:117] "RemoveContainer" containerID="8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54" Aug 04 22:55:44.518158 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.518142 2571 scope.go:117] "RemoveContainer" containerID="fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f" Aug 04 22:55:44.518385 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:55:44.518367 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f\": container with ID starting with fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f not found: ID does not exist" containerID="fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f" Aug 04 22:55:44.518448 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.518397 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f"} err="failed to get container status \"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f\": rpc error: code = NotFound desc = could not find container \"fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f\": container with ID starting with fcfc9de321106aa01025421d2e5221678b92202160a1b255cec46260ca571d6f not found: ID does not exist" Aug 04 22:55:44.518448 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.518422 2571 scope.go:117] "RemoveContainer" containerID="8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54" Aug 04 22:55:44.518653 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:55:44.518637 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54\": container with ID starting with 8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54 not found: ID does not exist" containerID="8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54" Aug 04 22:55:44.518694 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.518658 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54"} err="failed to get container status \"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54\": rpc error: code = NotFound desc = could not find container \"8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54\": container with ID starting with 8e36ad524815096f5306d7b9e6cb764c0cee0b80293851ddb858dc4c3f67ef54 not found: ID does not exist" Aug 04 22:55:44.522823 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.522803 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:44.526727 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:44.526704 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-8410bf38-openshift-default-78974dbdd4-wmbvc"] Aug 04 22:55:45.600858 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:55:45.600827 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" path="/var/lib/kubelet/pods/caf2f195-6889-4bbe-af7d-34de349c4c9f/volumes" Aug 04 22:57:37.448806 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.448775 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:57:37.449268 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.449078 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 22:57:37.449268 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.449091 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 22:57:37.449268 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.449154 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 22:57:37.449268 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.449164 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 22:57:37.451842 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.451825 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.454024 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.454002 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3c68233e-openshift-default-dockercfg-2pd4m\"" Aug 04 22:57:37.476484 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.476464 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:57:37.568773 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568738 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.568952 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.568952 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568801 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4njb\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.568952 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568886 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.569111 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568959 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.569111 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.568986 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.569111 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.569061 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.569111 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.569087 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.569235 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.569116 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670360 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670332 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670360 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670396 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670443 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670499 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670547 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670519 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4njb\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670748 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670748 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670612 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670748 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670642 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670898 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670750 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.670898 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.670800 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.671074 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.671021 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.671190 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.671080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.671248 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.671208 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.672853 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.672824 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.673134 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.673115 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.678351 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.678327 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.678469 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.678452 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4njb\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb\") pod \"e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.761579 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.761558 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:37.890635 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.890609 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:57:37.893154 ip-10-0-137-148 kubenswrapper[2571]: W0804 22:57:37.893123 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf14da558_53f9_412d_a676_9939723a0c74.slice/crio-75b65605c373d5a17d0611c42f0a9efb9e7879af0aa971b07ccabf353bee4b0f WatchSource:0}: Error finding container 75b65605c373d5a17d0611c42f0a9efb9e7879af0aa971b07ccabf353bee4b0f: Status 404 returned error can't find the container with id 75b65605c373d5a17d0611c42f0a9efb9e7879af0aa971b07ccabf353bee4b0f Aug 04 22:57:37.895059 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.895013 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:37.895139 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.895089 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:37.895139 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.895123 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:37.956121 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.956088 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerStarted","Data":"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470"} Aug 04 22:57:37.956229 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.956154 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerStarted","Data":"75b65605c373d5a17d0611c42f0a9efb9e7879af0aa971b07ccabf353bee4b0f"} Aug 04 22:57:37.976999 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:37.976955 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podStartSLOduration=0.976937694 podStartE2EDuration="976.937694ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 22:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 22:57:37.975833163 +0000 UTC m=+2200.919339397" watchObservedRunningTime="2026-08-04 22:57:37.976937694 +0000 UTC m=+2200.920443927" Aug 04 22:57:38.761916 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:38.761884 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:38.766469 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:38.766443 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:38.960667 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:38.960636 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:38.961431 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:38.961410 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:53.961913 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:53.961876 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:53.962430 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:53.961950 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:57:54.024076 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.024029 2571 generic.go:356] "Generic (PLEG): container finished" podID="f14da558-53f9-412d-a676-9939723a0c74" containerID="2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470" exitCode=0 Aug 04 22:57:54.024225 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.024073 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerDied","Data":"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470"} Aug 04 22:57:54.024525 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.024506 2571 scope.go:117] "RemoveContainer" containerID="2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470" Aug 04 22:57:54.031051 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.031000 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:54.031170 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.031103 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:54.031170 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:54.031144 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 22:57:55.034942 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:55.034906 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerStarted","Data":"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c"} Aug 04 22:57:55.762258 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:55.762217 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:57:55.762467 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:55.762440 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:55.762683 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:55.762659 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:57:56.762641 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:56.762600 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:56.763248 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:56.763216 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:57:57.762533 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:57.762490 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:57.762744 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:57.762556 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:57:58.762795 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:58.762747 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:58.763299 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:58.762829 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:57:59.762684 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:59.762645 2571 patch_prober.go:28] interesting pod/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Aug 04 22:57:59.762886 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:57:59.762724 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Aug 04 22:58:00.765906 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:00.765875 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:58:00.766390 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:00.766297 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:58:00.766976 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:00.766954 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:58:23.121186 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:23.121113 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:58:23.121710 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:23.121310 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" containerID="cri-o://d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c" gracePeriod=30 Aug 04 22:58:28.357846 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.357819 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:58:28.498514 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498487 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.498709 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498552 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.498709 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498572 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.498709 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498599 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.498709 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498651 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4njb\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.498709 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498674 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.499026 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498712 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.499026 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498761 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.499026 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498795 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs\") pod \"f14da558-53f9-412d-a676-9939723a0c74\" (UID: \"f14da558-53f9-412d-a676-9939723a0c74\") " Aug 04 22:58:28.499026 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.498975 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:58:28.499026 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.499009 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 22:58:28.499313 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.499108 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.499313 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.499240 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:58:28.499313 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.499254 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:58:28.499459 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.499328 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data" (OuterVolumeSpecName: "istio-data") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:58:28.500990 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.500958 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 22:58:28.501128 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.501015 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token" (OuterVolumeSpecName: "istio-token") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:58:28.501128 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.501017 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 22:58:28.501128 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.501059 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb" (OuterVolumeSpecName: "kube-api-access-z4njb") pod "f14da558-53f9-412d-a676-9939723a0c74" (UID: "f14da558-53f9-412d-a676-9939723a0c74"). InnerVolumeSpecName "kube-api-access-z4njb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 22:58:28.600185 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600145 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600185 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600180 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f14da558-53f9-412d-a676-9939723a0c74-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600185 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600190 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600185 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600198 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600206 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4njb\" (UniqueName: \"kubernetes.io/projected/f14da558-53f9-412d-a676-9939723a0c74-kube-api-access-z4njb\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600215 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f14da558-53f9-412d-a676-9939723a0c74-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600223 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:28.600413 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:28.600231 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f14da558-53f9-412d-a676-9939723a0c74-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 22:58:29.174852 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.174814 2571 generic.go:356] "Generic (PLEG): container finished" podID="f14da558-53f9-412d-a676-9939723a0c74" containerID="d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c" exitCode=0 Aug 04 22:58:29.175010 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.174858 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" Aug 04 22:58:29.175010 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.174889 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerDied","Data":"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c"} Aug 04 22:58:29.175010 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.174935 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb" event={"ID":"f14da558-53f9-412d-a676-9939723a0c74","Type":"ContainerDied","Data":"75b65605c373d5a17d0611c42f0a9efb9e7879af0aa971b07ccabf353bee4b0f"} Aug 04 22:58:29.175010 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.174959 2571 scope.go:117] "RemoveContainer" containerID="d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c" Aug 04 22:58:29.183819 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.183801 2571 scope.go:117] "RemoveContainer" containerID="2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470" Aug 04 22:58:29.191000 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.190984 2571 scope.go:117] "RemoveContainer" containerID="d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c" Aug 04 22:58:29.191278 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:58:29.191256 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c\": container with ID starting with d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c not found: ID does not exist" containerID="d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c" Aug 04 22:58:29.191336 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.191288 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c"} err="failed to get container status \"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c\": rpc error: code = NotFound desc = could not find container \"d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c\": container with ID starting with d5aa72f54a9f3b90b2a11cfdd3c1a94e2aa417095233dba58a50f8a44366e01c not found: ID does not exist" Aug 04 22:58:29.191336 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.191312 2571 scope.go:117] "RemoveContainer" containerID="2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470" Aug 04 22:58:29.191561 ip-10-0-137-148 kubenswrapper[2571]: E0804 22:58:29.191543 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470\": container with ID starting with 2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470 not found: ID does not exist" containerID="2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470" Aug 04 22:58:29.191613 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.191564 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470"} err="failed to get container status \"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470\": rpc error: code = NotFound desc = could not find container \"2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470\": container with ID starting with 2c94c7d72d115bd7e210f3454691b86a2ba11d205a0c349af2be2ea3210e5470 not found: ID does not exist" Aug 04 22:58:29.194911 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.194891 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:58:29.197572 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.197553 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3c68233e-openshift-default-bc978fb7b-d2wjb"] Aug 04 22:58:29.601509 ip-10-0-137-148 kubenswrapper[2571]: I0804 22:58:29.601474 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f14da558-53f9-412d-a676-9939723a0c74" path="/var/lib/kubelet/pods/f14da558-53f9-412d-a676-9939723a0c74/volumes" Aug 04 23:04:34.703962 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.703874 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704359 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704378 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704394 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704403 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704425 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704435 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="caf2f195-6889-4bbe-af7d-34de349c4c9f" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704501 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.704521 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.704512 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f14da558-53f9-412d-a676-9939723a0c74" containerName="istio-proxy" Aug 04 23:04:34.706501 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.706479 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.708624 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.708604 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2d20db-openshift-default-dockercfg-zfvhf\"" Aug 04 23:04:34.718160 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.718142 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:04:34.753690 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753663 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zjxh\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753810 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753696 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753810 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753715 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753810 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753730 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753810 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753785 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753863 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753935 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.753963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.753960 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.854819 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854784 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjxh\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.854819 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854821 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854842 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854857 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854872 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854917 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.854947 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855352 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855172 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855352 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855234 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855352 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855326 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855502 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855366 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855502 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855528 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.855601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.855595 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.857284 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.857254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.857462 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.857445 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.862152 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.862128 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:34.862331 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:34.862313 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjxh\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh\") pod \"e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:35.016951 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.016875 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:35.145738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.145645 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:04:35.148461 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:04:35.148435 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b0b5df_6b74_4121_bbcb_8f973166b05c.slice/crio-f115fc587659d375705088f9023a4a24eeffe39b6e520688339ac866368bd2a0 WatchSource:0}: Error finding container f115fc587659d375705088f9023a4a24eeffe39b6e520688339ac866368bd2a0: Status 404 returned error can't find the container with id f115fc587659d375705088f9023a4a24eeffe39b6e520688339ac866368bd2a0 Aug 04 23:04:35.150875 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.150846 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:35.150938 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.150908 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:35.150974 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.150938 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:35.601067 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.601016 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerStarted","Data":"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e"} Aug 04 23:04:35.601223 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.601084 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerStarted","Data":"f115fc587659d375705088f9023a4a24eeffe39b6e520688339ac866368bd2a0"} Aug 04 23:04:35.621417 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:35.621376 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podStartSLOduration=1.621361618 podStartE2EDuration="1.621361618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:04:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:04:35.620317505 +0000 UTC m=+2618.563823737" watchObservedRunningTime="2026-08-04 23:04:35.621361618 +0000 UTC m=+2618.564867849" Aug 04 23:04:36.017516 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:36.017488 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:36.022636 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:36.022616 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:36.605660 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:36.605629 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:36.606510 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:36.606493 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:51.606828 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.606792 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:51.607306 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.606851 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:51.668523 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.668494 2571 generic.go:356] "Generic (PLEG): container finished" podID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerID="5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e" exitCode=0 Aug 04 23:04:51.668667 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.668568 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerDied","Data":"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e"} Aug 04 23:04:51.668922 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.668908 2571 scope.go:117] "RemoveContainer" containerID="5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e" Aug 04 23:04:51.669894 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.669867 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:51.670072 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.669953 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:51.670072 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:51.670004 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:04:52.677767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:52.677736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerStarted","Data":"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5"} Aug 04 23:04:53.017334 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:53.017299 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:53.017500 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:53.017474 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:53.017546 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:53.017523 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:54.017210 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:54.017171 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:54.017749 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:54.017248 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:55.018179 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:55.018145 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:55.018560 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:55.018198 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:56.018270 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:56.018235 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:56.018642 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:56.018290 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:57.017564 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:57.017524 2571 patch_prober.go:28] interesting pod/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Aug 04 23:04:57.017760 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:57.017586 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Aug 04 23:04:58.021638 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:58.021610 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:58.022056 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:58.022016 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:04:58.022767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:04:58.022747 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:05:10.791574 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.791537 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:05:10.793963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.793939 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.796386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.796361 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-549992\"/\"openshift-service-ca.crt\"" Aug 04 23:05:10.797110 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.797090 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-549992\"/\"default-dockercfg-525fw\"" Aug 04 23:05:10.797206 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.797144 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-549992\"/\"test-model-549992-kserve-self-signed-certs\"" Aug 04 23:05:10.797206 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.797090 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-549992\"/\"kube-root-ca.crt\"" Aug 04 23:05:10.804020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.803996 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:05:10.877071 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877019 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.877198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877082 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.877198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877156 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.877198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877175 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.877198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877190 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.877386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.877219 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978447 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978409 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978447 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978457 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978701 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978653 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978752 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978736 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978791 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978769 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978826 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978880 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978855 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.978958 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.978832 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.979188 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.979168 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.980924 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.980903 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.981327 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.981307 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:10.986640 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:10.986616 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt\") pod \"test-model-549992-kserve-9ccf96769-db78w\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:11.104751 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:11.104674 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:11.226925 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:11.226902 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:05:11.229246 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:05:11.229222 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf16add60_9570_4b77_8b2c_29fe1a940a57.slice/crio-ca2eb84f9c8e7aaba151256b80579d7ecd16d8f2d0934c5da67ece6ce8f395d0 WatchSource:0}: Error finding container ca2eb84f9c8e7aaba151256b80579d7ecd16d8f2d0934c5da67ece6ce8f395d0: Status 404 returned error can't find the container with id ca2eb84f9c8e7aaba151256b80579d7ecd16d8f2d0934c5da67ece6ce8f395d0 Aug 04 23:05:11.753811 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:11.753772 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" event={"ID":"f16add60-9570-4b77-8b2c-29fe1a940a57","Type":"ContainerStarted","Data":"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066"} Aug 04 23:05:11.753811 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:11.753819 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" event={"ID":"f16add60-9570-4b77-8b2c-29fe1a940a57","Type":"ContainerStarted","Data":"ca2eb84f9c8e7aaba151256b80579d7ecd16d8f2d0934c5da67ece6ce8f395d0"} Aug 04 23:05:11.770726 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:11.770678 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" podStartSLOduration=1.770659963 podStartE2EDuration="1.770659963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:05:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:05:11.769440871 +0000 UTC m=+2654.712947132" watchObservedRunningTime="2026-08-04 23:05:11.770659963 +0000 UTC m=+2654.714166196" Aug 04 23:05:21.105793 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:21.105761 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:21.106265 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:21.105810 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:21.118304 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:21.118277 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:05:21.805235 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:05:21.805209 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:06:22.207976 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:22.207940 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:06:37.226895 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.226839 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" podUID="f16add60-9570-4b77-8b2c-29fe1a940a57" containerName="main" containerID="cri-o://157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066" gracePeriod=45 Aug 04 23:06:37.469510 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.469486 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:06:37.582167 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582094 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582167 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582128 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582167 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582159 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582432 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582207 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582432 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582229 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582432 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582252 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs\") pod \"f16add60-9570-4b77-8b2c-29fe1a940a57\" (UID: \"f16add60-9570-4b77-8b2c-29fe1a940a57\") " Aug 04 23:06:37.582570 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582400 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home" (OuterVolumeSpecName: "home") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:06:37.582570 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.582462 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:06:37.589183 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.583026 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache" (OuterVolumeSpecName: "model-cache") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:06:37.589183 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.583145 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-tmp-dir\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:37.589183 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.583270 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-home\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:37.589646 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.589530 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm" (OuterVolumeSpecName: "dshm") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:06:37.589646 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.589609 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt" (OuterVolumeSpecName: "kube-api-access-dk8jt") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "kube-api-access-dk8jt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:06:37.589728 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.589639 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f16add60-9570-4b77-8b2c-29fe1a940a57" (UID: "f16add60-9570-4b77-8b2c-29fe1a940a57"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 23:06:37.684350 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.684326 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dk8jt\" (UniqueName: \"kubernetes.io/projected/f16add60-9570-4b77-8b2c-29fe1a940a57-kube-api-access-dk8jt\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:37.684350 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.684346 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-model-cache\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:37.684350 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.684355 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f16add60-9570-4b77-8b2c-29fe1a940a57-tls-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:37.684535 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:37.684363 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f16add60-9570-4b77-8b2c-29fe1a940a57-dshm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:06:38.129519 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.129485 2571 generic.go:356] "Generic (PLEG): container finished" podID="f16add60-9570-4b77-8b2c-29fe1a940a57" containerID="157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066" exitCode=0 Aug 04 23:06:38.129706 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.129533 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" Aug 04 23:06:38.129706 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.129539 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" event={"ID":"f16add60-9570-4b77-8b2c-29fe1a940a57","Type":"ContainerDied","Data":"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066"} Aug 04 23:06:38.129706 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.129580 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w" event={"ID":"f16add60-9570-4b77-8b2c-29fe1a940a57","Type":"ContainerDied","Data":"ca2eb84f9c8e7aaba151256b80579d7ecd16d8f2d0934c5da67ece6ce8f395d0"} Aug 04 23:06:38.129706 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.129601 2571 scope.go:117] "RemoveContainer" containerID="157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066" Aug 04 23:06:38.137550 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.137533 2571 scope.go:117] "RemoveContainer" containerID="157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066" Aug 04 23:06:38.137788 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:06:38.137769 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066\": container with ID starting with 157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066 not found: ID does not exist" containerID="157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066" Aug 04 23:06:38.137833 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.137796 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066"} err="failed to get container status \"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066\": rpc error: code = NotFound desc = could not find container \"157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066\": container with ID starting with 157d4507e1206add47b65e3601e2161a2687055150d25b7c8be36726428bd066 not found: ID does not exist" Aug 04 23:06:38.145071 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.145030 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:06:38.148946 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:38.148923 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-549992/test-model-549992-kserve-9ccf96769-db78w"] Aug 04 23:06:39.601365 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:39.601329 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16add60-9570-4b77-8b2c-29fe1a940a57" path="/var/lib/kubelet/pods/f16add60-9570-4b77-8b2c-29fe1a940a57/volumes" Aug 04 23:06:58.313795 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:58.313762 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:06:58.314308 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:06:58.314026 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" containerID="cri-o://a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5" gracePeriod=30 Aug 04 23:07:03.562970 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.562943 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:07:03.695076 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.694992 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695076 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695025 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695076 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695061 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695088 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695114 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695141 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695156 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695289 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695298 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:07:03.695333 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695334 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695655 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695372 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zjxh\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh\") pod \"24b0b5df-6b74-4121-bbcb-8f973166b05c\" (UID: \"24b0b5df-6b74-4121-bbcb-8f973166b05c\") " Aug 04 23:07:03.695655 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695462 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data" (OuterVolumeSpecName: "istio-data") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:07:03.695655 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695527 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:07:03.695655 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695563 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:07:03.695863 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695684 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.695863 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695704 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.695863 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695718 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.695863 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695731 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24b0b5df-6b74-4121-bbcb-8f973166b05c-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.695863 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.695743 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.697399 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.697363 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:07:03.697504 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.697436 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:07:03.697504 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.697458 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token" (OuterVolumeSpecName: "istio-token") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:07:03.697590 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.697510 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh" (OuterVolumeSpecName: "kube-api-access-5zjxh") pod "24b0b5df-6b74-4121-bbcb-8f973166b05c" (UID: "24b0b5df-6b74-4121-bbcb-8f973166b05c"). InnerVolumeSpecName "kube-api-access-5zjxh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:07:03.796420 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.796395 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.796420 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.796416 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.796420 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.796425 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24b0b5df-6b74-4121-bbcb-8f973166b05c-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:03.796592 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:03.796433 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zjxh\" (UniqueName: \"kubernetes.io/projected/24b0b5df-6b74-4121-bbcb-8f973166b05c-kube-api-access-5zjxh\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:07:04.235419 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.235380 2571 generic.go:356] "Generic (PLEG): container finished" podID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerID="a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5" exitCode=0 Aug 04 23:07:04.235558 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.235448 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" Aug 04 23:07:04.235558 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.235465 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerDied","Data":"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5"} Aug 04 23:07:04.235558 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.235527 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x" event={"ID":"24b0b5df-6b74-4121-bbcb-8f973166b05c","Type":"ContainerDied","Data":"f115fc587659d375705088f9023a4a24eeffe39b6e520688339ac866368bd2a0"} Aug 04 23:07:04.235558 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.235553 2571 scope.go:117] "RemoveContainer" containerID="a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5" Aug 04 23:07:04.244522 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.244503 2571 scope.go:117] "RemoveContainer" containerID="5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e" Aug 04 23:07:04.252567 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.252501 2571 scope.go:117] "RemoveContainer" containerID="a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5" Aug 04 23:07:04.253851 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:07:04.253832 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5\": container with ID starting with a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5 not found: ID does not exist" containerID="a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5" Aug 04 23:07:04.253942 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.253857 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5"} err="failed to get container status \"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5\": rpc error: code = NotFound desc = could not find container \"a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5\": container with ID starting with a0977dae1d86bf8bc8942d7f6e3ae9321cfa4bfbc0618fd45747f793d229dbf5 not found: ID does not exist" Aug 04 23:07:04.253942 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.253875 2571 scope.go:117] "RemoveContainer" containerID="5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e" Aug 04 23:07:04.254220 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:07:04.254202 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e\": container with ID starting with 5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e not found: ID does not exist" containerID="5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e" Aug 04 23:07:04.254305 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.254223 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e"} err="failed to get container status \"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e\": rpc error: code = NotFound desc = could not find container \"5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e\": container with ID starting with 5a78350c51f3f7a5b47ef964b43b10e45616e43a6fd776b438734a05f4b5659e not found: ID does not exist" Aug 04 23:07:04.256599 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.256575 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:07:04.260476 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:04.260457 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2d20db-openshift-default-b759889b4-gnv5x"] Aug 04 23:07:05.600445 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:05.600413 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" path="/var/lib/kubelet/pods/24b0b5df-6b74-4121-bbcb-8f973166b05c/volumes" Aug 04 23:07:27.853330 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853249 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853524 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853535 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853552 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16add60-9570-4b77-8b2c-29fe1a940a57" containerName="main" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853558 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16add60-9570-4b77-8b2c-29fe1a940a57" containerName="main" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853604 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853611 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:07:27.853702 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.853619 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16add60-9570-4b77-8b2c-29fe1a940a57" containerName="main" Aug 04 23:07:27.856475 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.856457 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.858929 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.858908 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-25c344-openshift-default-dockercfg-l2g7l\"" Aug 04 23:07:27.877805 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.877781 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:07:27.977365 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977327 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977365 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977364 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977382 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8zjm\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977404 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977442 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977464 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977502 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:27.977554 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:27.977520 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078319 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078283 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078319 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078320 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078350 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8zjm\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078381 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078414 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078448 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078472 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078501 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078528 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078915 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078733 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078915 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078845 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.078915 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078882 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.079060 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.078928 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.079108 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.079094 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.080677 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.080657 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.080913 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.080894 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.085874 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.085854 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.086097 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.086080 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8zjm\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm\") pod \"e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.165421 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.165349 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:28.295716 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.295691 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:07:28.297817 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:07:28.297791 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod720237c9_cbf6_481c_ba9e_f95d0a88c9b5.slice/crio-cc605c7d7ad2bd9138209153aa604a6bd4660f0a2092177d0a4c055c7f302681 WatchSource:0}: Error finding container cc605c7d7ad2bd9138209153aa604a6bd4660f0a2092177d0a4c055c7f302681: Status 404 returned error can't find the container with id cc605c7d7ad2bd9138209153aa604a6bd4660f0a2092177d0a4c055c7f302681 Aug 04 23:07:28.299905 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.299869 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:28.299977 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.299952 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:28.300014 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.299991 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:28.329089 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:28.329066 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerStarted","Data":"cc605c7d7ad2bd9138209153aa604a6bd4660f0a2092177d0a4c055c7f302681"} Aug 04 23:07:29.334816 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:29.334780 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerStarted","Data":"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48"} Aug 04 23:07:29.356030 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:29.355975 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podStartSLOduration=2.355961951 podStartE2EDuration="2.355961951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:07:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:07:29.35464319 +0000 UTC m=+2792.298149445" watchObservedRunningTime="2026-08-04 23:07:29.355961951 +0000 UTC m=+2792.299468225" Aug 04 23:07:30.165616 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:30.165584 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:30.170657 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:30.170631 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:30.338683 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:30.338654 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:30.339507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:30.339488 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:45.584436 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:45.584403 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": read tcp 10.133.0.2:45728->10.133.0.27:15021: read: connection reset by peer" start-of-body= Aug 04 23:07:45.584847 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:45.584470 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": read tcp 10.133.0.2:45728->10.133.0.27:15021: read: connection reset by peer" Aug 04 23:07:46.410661 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.410628 2571 generic.go:356] "Generic (PLEG): container finished" podID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerID="d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48" exitCode=0 Aug 04 23:07:46.410828 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.410699 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerDied","Data":"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48"} Aug 04 23:07:46.414066 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.411431 2571 scope.go:117] "RemoveContainer" containerID="d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48" Aug 04 23:07:46.417012 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.416981 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:46.417161 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.417084 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:46.417161 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:46.417140 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:07:47.416376 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:47.416343 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerStarted","Data":"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0"} Aug 04 23:07:48.166808 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:48.166764 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:48.167014 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:48.166892 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Aug 04 23:07:48.167014 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:48.166932 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Aug 04 23:07:49.166507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:49.166471 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Aug 04 23:07:49.167101 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:49.167065 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Aug 04 23:07:50.166497 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:50.166455 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Aug 04 23:07:50.166764 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:50.166732 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Aug 04 23:07:51.166397 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:51.166360 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Aug 04 23:07:51.166694 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:51.166661 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Aug 04 23:07:52.166162 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:52.166125 2571 patch_prober.go:28] interesting pod/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Aug 04 23:07:52.166557 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:52.166182 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Aug 04 23:07:53.169985 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:53.169954 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:53.170385 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:53.170223 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:07:53.170935 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:07:53.170914 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:08:44.012101 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:44.012059 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:08:44.012572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:44.012314 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" containerID="cri-o://5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0" gracePeriod=30 Aug 04 23:08:49.246939 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.246913 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:08:49.313211 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313149 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313211 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313191 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313377 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313216 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313377 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313256 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313377 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313281 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313377 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313299 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8zjm\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313377 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313330 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313622 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313389 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313622 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313417 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data\") pod \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\" (UID: \"720237c9-cbf6-481c-ba9e-f95d0a88c9b5\") " Aug 04 23:08:49.313622 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313422 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:08:49.313767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313590 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:08:49.313767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313672 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:08:49.313767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313703 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:08:49.313767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313752 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.313958 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313797 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.313958 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.313844 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:08:49.315475 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.315448 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:08:49.315576 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.315552 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm" (OuterVolumeSpecName: "kube-api-access-n8zjm") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "kube-api-access-n8zjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:08:49.315745 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.315721 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:08:49.315745 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.315728 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "720237c9-cbf6-481c-ba9e-f95d0a88c9b5" (UID: "720237c9-cbf6-481c-ba9e-f95d0a88c9b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:08:49.414401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414372 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414396 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414405 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414597 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414412 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414597 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414420 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414597 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414428 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8zjm\" (UniqueName: \"kubernetes.io/projected/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-kube-api-access-n8zjm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.414597 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.414438 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/720237c9-cbf6-481c-ba9e-f95d0a88c9b5-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:08:49.666690 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.666654 2571 generic.go:356] "Generic (PLEG): container finished" podID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerID="5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0" exitCode=0 Aug 04 23:08:49.666833 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.666711 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" Aug 04 23:08:49.666833 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.666710 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerDied","Data":"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0"} Aug 04 23:08:49.666904 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.666854 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq" event={"ID":"720237c9-cbf6-481c-ba9e-f95d0a88c9b5","Type":"ContainerDied","Data":"cc605c7d7ad2bd9138209153aa604a6bd4660f0a2092177d0a4c055c7f302681"} Aug 04 23:08:49.666904 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.666880 2571 scope.go:117] "RemoveContainer" containerID="5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0" Aug 04 23:08:49.677202 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.677090 2571 scope.go:117] "RemoveContainer" containerID="d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48" Aug 04 23:08:49.684537 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.684514 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:08:49.684746 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.684730 2571 scope.go:117] "RemoveContainer" containerID="5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0" Aug 04 23:08:49.684997 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:08:49.684979 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0\": container with ID starting with 5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0 not found: ID does not exist" containerID="5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0" Aug 04 23:08:49.685052 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.685007 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0"} err="failed to get container status \"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0\": rpc error: code = NotFound desc = could not find container \"5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0\": container with ID starting with 5a81d2d42c251e665c9ce8c6fc3d6d7fe454dc47adb899e2e42b1db05adc78e0 not found: ID does not exist" Aug 04 23:08:49.685052 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.685025 2571 scope.go:117] "RemoveContainer" containerID="d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48" Aug 04 23:08:49.685280 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:08:49.685266 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48\": container with ID starting with d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48 not found: ID does not exist" containerID="d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48" Aug 04 23:08:49.685322 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.685285 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48"} err="failed to get container status \"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48\": rpc error: code = NotFound desc = could not find container \"d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48\": container with ID starting with d2eefd012cdd0ec8293cfa5c43fea2001b5ef43b4da2e4874eaa96088ed47f48 not found: ID does not exist" Aug 04 23:08:49.690365 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:49.690344 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-25c344-openshift-default-67d64cb684-v4ssq"] Aug 04 23:08:51.600913 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:08:51.600882 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" path="/var/lib/kubelet/pods/720237c9-cbf6-481c-ba9e-f95d0a88c9b5/volumes" Aug 04 23:09:13.430978 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.430943 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431226 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431238 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b0b5df-6b74-4121-bbcb-8f973166b05c" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431245 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431251 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431259 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431265 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431314 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.431341 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.431321 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="720237c9-cbf6-481c-ba9e-f95d0a88c9b5" containerName="istio-proxy" Aug 04 23:09:13.433211 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.433192 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.435514 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.435493 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-55909a-openshift-default-dockercfg-g94zt\"" Aug 04 23:09:13.444239 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.444216 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:09:13.602213 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602184 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602381 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602226 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602381 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602284 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602381 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602306 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602381 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602357 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602531 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602387 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602531 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602435 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602531 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b78z4\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.602531 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.602494 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703203 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703171 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703335 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703275 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703335 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703309 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b78z4\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703454 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703335 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703454 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703360 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703454 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703427 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703590 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703525 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703590 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703567 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703703 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703603 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703788 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703603 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703788 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703776 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.703893 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.703813 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.704084 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.704062 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.704273 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.704254 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.705825 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.705806 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.706109 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.706092 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.710614 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.710592 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.710858 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.710843 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b78z4\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4\") pod \"e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.742943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.742927 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:13.868571 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.868545 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:09:13.870879 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:09:13.870853 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3d4ed4_eac5_48f5_adff_399f3bdc2a15.slice/crio-691cbc5a3bbe89f756f478606dc73d767a6f7bb692b5c2246e5c02e67415bb49 WatchSource:0}: Error finding container 691cbc5a3bbe89f756f478606dc73d767a6f7bb692b5c2246e5c02e67415bb49: Status 404 returned error can't find the container with id 691cbc5a3bbe89f756f478606dc73d767a6f7bb692b5c2246e5c02e67415bb49 Aug 04 23:09:13.873016 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.872982 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:13.873129 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.873061 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:13.873129 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:13.873093 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:14.763508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:14.763476 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerStarted","Data":"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a"} Aug 04 23:09:14.763950 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:14.763529 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerStarted","Data":"691cbc5a3bbe89f756f478606dc73d767a6f7bb692b5c2246e5c02e67415bb49"} Aug 04 23:09:14.783642 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:14.783593 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podStartSLOduration=1.783578036 podStartE2EDuration="1.783578036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:09:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:09:14.781471527 +0000 UTC m=+2897.724977760" watchObservedRunningTime="2026-08-04 23:09:14.783578036 +0000 UTC m=+2897.727084305" Aug 04 23:09:15.744091 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:15.744064 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:15.749436 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:15.749413 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:15.768008 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:15.767982 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:15.768922 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:15.768897 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:30.768917 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.768880 2571 patch_prober.go:28] interesting pod/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Aug 04 23:09:30.769351 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.768945 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Aug 04 23:09:30.832148 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.832112 2571 generic.go:356] "Generic (PLEG): container finished" podID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerID="48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a" exitCode=0 Aug 04 23:09:30.832307 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.832185 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerDied","Data":"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a"} Aug 04 23:09:30.832556 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.832542 2571 scope.go:117] "RemoveContainer" containerID="48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a" Aug 04 23:09:30.833631 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.833603 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:30.833727 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.833684 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:30.833787 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:30.833730 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:09:31.838102 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:31.838064 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerStarted","Data":"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c"} Aug 04 23:09:32.743696 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:32.743661 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:32.744080 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:32.744056 2571 patch_prober.go:28] interesting pod/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Aug 04 23:09:32.744273 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:32.744247 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Aug 04 23:09:33.743894 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:33.743858 2571 patch_prober.go:28] interesting pod/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Aug 04 23:09:33.744493 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:33.744463 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Aug 04 23:09:34.757489 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:34.757448 2571 patch_prober.go:28] interesting pod/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Aug 04 23:09:34.757994 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:34.757519 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Aug 04 23:09:35.744008 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:35.743972 2571 patch_prober.go:28] interesting pod/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Aug 04 23:09:35.744314 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:35.744285 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Aug 04 23:09:36.771519 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:36.771487 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:36.771938 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:36.771911 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:36.772730 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:36.772710 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:09:47.193434 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.193400 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:09:47.195472 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.195453 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.198508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.198480 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"default-dockercfg-g4dlq\"" Aug 04 23:09:47.198508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.198480 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"auth-test-model-ec06a8-kserve-self-signed-certs\"" Aug 04 23:09:47.198657 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.198528 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"kube-root-ca.crt\"" Aug 04 23:09:47.198657 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.198536 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"openshift-service-ca.crt\"" Aug 04 23:09:47.206187 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.206164 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:09:47.292734 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292709 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.292856 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292814 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.292908 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292856 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.292908 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292876 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.293011 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292954 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj6hr\" (UniqueName: \"kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.293011 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.292994 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.393952 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.393915 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394140 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.393974 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394140 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394005 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394140 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394066 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj6hr\" (UniqueName: \"kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394140 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394103 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394347 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394147 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394432 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394486 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394462 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.394486 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.394476 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.396262 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.396238 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.396502 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.396485 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.401753 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.401732 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj6hr\" (UniqueName: \"kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr\") pod \"auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.504514 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.504485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:47.634766 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.634725 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:09:47.636660 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:09:47.636635 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f82b78e_663d_4faa_b3b0_4680786c9d44.slice/crio-6a06b0da10eb5bf9f7df85428e8d099f428dcb1f7c6741060f51ebcff7e3ccbc WatchSource:0}: Error finding container 6a06b0da10eb5bf9f7df85428e8d099f428dcb1f7c6741060f51ebcff7e3ccbc: Status 404 returned error can't find the container with id 6a06b0da10eb5bf9f7df85428e8d099f428dcb1f7c6741060f51ebcff7e3ccbc Aug 04 23:09:47.914543 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.914448 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" event={"ID":"4f82b78e-663d-4faa-b3b0-4680786c9d44","Type":"ContainerStarted","Data":"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f"} Aug 04 23:09:47.914543 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.914493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" event={"ID":"4f82b78e-663d-4faa-b3b0-4680786c9d44","Type":"ContainerStarted","Data":"6a06b0da10eb5bf9f7df85428e8d099f428dcb1f7c6741060f51ebcff7e3ccbc"} Aug 04 23:09:47.931710 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:47.931665 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" podStartSLOduration=0.931651849 podStartE2EDuration="931.651849ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:09:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:09:47.931079392 +0000 UTC m=+2930.874585622" watchObservedRunningTime="2026-08-04 23:09:47.931651849 +0000 UTC m=+2930.875158080" Aug 04 23:09:57.505611 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:57.505574 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:57.506097 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:57.505627 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:57.518096 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:57.518075 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:09:57.965755 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:09:57.965727 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:13:17.470171 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:17.470140 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:13:32.488771 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.488715 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" podUID="4f82b78e-663d-4faa-b3b0-4680786c9d44" containerName="main" containerID="cri-o://560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f" gracePeriod=45 Aug 04 23:13:32.733181 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.733159 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:13:32.819624 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.819550 2571 generic.go:356] "Generic (PLEG): container finished" podID="4f82b78e-663d-4faa-b3b0-4680786c9d44" containerID="560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f" exitCode=0 Aug 04 23:13:32.819624 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.819592 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" Aug 04 23:13:32.819770 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.819592 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" event={"ID":"4f82b78e-663d-4faa-b3b0-4680786c9d44","Type":"ContainerDied","Data":"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f"} Aug 04 23:13:32.819770 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.819697 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw" event={"ID":"4f82b78e-663d-4faa-b3b0-4680786c9d44","Type":"ContainerDied","Data":"6a06b0da10eb5bf9f7df85428e8d099f428dcb1f7c6741060f51ebcff7e3ccbc"} Aug 04 23:13:32.819770 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.819715 2571 scope.go:117] "RemoveContainer" containerID="560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f" Aug 04 23:13:32.827375 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.827358 2571 scope.go:117] "RemoveContainer" containerID="560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f" Aug 04 23:13:32.827608 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:13:32.827575 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f\": container with ID starting with 560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f not found: ID does not exist" containerID="560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f" Aug 04 23:13:32.827708 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.827606 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f"} err="failed to get container status \"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f\": rpc error: code = NotFound desc = could not find container \"560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f\": container with ID starting with 560d8e02b37c88f5baa6acdcc970a4e3fafebce1a30b27de66a0a537f73a2f5f not found: ID does not exist" Aug 04 23:13:32.849832 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.849815 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.849921 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.849850 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.849921 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.849874 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.849921 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.849903 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj6hr\" (UniqueName: \"kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.850067 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.849945 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.850067 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.850012 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir\") pod \"4f82b78e-663d-4faa-b3b0-4680786c9d44\" (UID: \"4f82b78e-663d-4faa-b3b0-4680786c9d44\") " Aug 04 23:13:32.850171 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.850136 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home" (OuterVolumeSpecName: "home") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:13:32.850268 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.850252 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-home\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:32.850366 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.850316 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:13:32.850366 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.850332 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:13:32.851973 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.851949 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 23:13:32.852073 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.852017 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm" (OuterVolumeSpecName: "dshm") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:13:32.852073 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.852018 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr" (OuterVolumeSpecName: "kube-api-access-sj6hr") pod "4f82b78e-663d-4faa-b3b0-4680786c9d44" (UID: "4f82b78e-663d-4faa-b3b0-4680786c9d44"). InnerVolumeSpecName "kube-api-access-sj6hr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:13:32.950801 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.950774 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj6hr\" (UniqueName: \"kubernetes.io/projected/4f82b78e-663d-4faa-b3b0-4680786c9d44-kube-api-access-sj6hr\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:32.950801 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.950797 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-model-cache\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:32.950801 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.950807 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-tmp-dir\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:32.950981 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.950814 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f82b78e-663d-4faa-b3b0-4680786c9d44-tls-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:32.950981 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:32.950822 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f82b78e-663d-4faa-b3b0-4680786c9d44-dshm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:13:33.139500 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:33.139475 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:13:33.142986 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:33.142964 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/auth-test-model-ec06a8-kserve-78f7bf8cff-b6pxw"] Aug 04 23:13:33.601877 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:33.601843 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f82b78e-663d-4faa-b3b0-4680786c9d44" path="/var/lib/kubelet/pods/4f82b78e-663d-4faa-b3b0-4680786c9d44/volumes" Aug 04 23:13:59.493253 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.493220 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:13:59.493650 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.493499 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f82b78e-663d-4faa-b3b0-4680786c9d44" containerName="main" Aug 04 23:13:59.493650 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.493509 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f82b78e-663d-4faa-b3b0-4680786c9d44" containerName="main" Aug 04 23:13:59.493650 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.493577 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f82b78e-663d-4faa-b3b0-4680786c9d44" containerName="main" Aug 04 23:13:59.497552 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.497534 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.499619 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.499596 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-55909a\"/\"kube-root-ca.crt\"" Aug 04 23:13:59.500472 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.500444 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-55909a\"/\"default-dockercfg-ksblg\"" Aug 04 23:13:59.500562 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.500444 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-55909a\"/\"sub-test-model-55909a-kserve-self-signed-certs\"" Aug 04 23:13:59.500562 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.500454 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-55909a\"/\"openshift-service-ca.crt\"" Aug 04 23:13:59.505866 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.505844 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:13:59.552015 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.551988 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.552129 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.552026 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.552129 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.552085 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.552129 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.552101 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.552246 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.552203 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.552246 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.552230 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxcpl\" (UniqueName: \"kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653347 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653307 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653363 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxcpl\" (UniqueName: \"kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653395 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653447 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653496 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653710 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653520 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653765 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653712 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653765 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653734 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.653868 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.653827 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.655873 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.655847 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.656153 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.656134 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.662483 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.662464 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxcpl\" (UniqueName: \"kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl\") pod \"sub-test-model-55909a-kserve-6f8c8fc685-qwv2t\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.807921 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.807841 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:13:59.936093 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:13:59.933931 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:14:00.933090 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:00.933052 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" event={"ID":"3099fcf6-f44d-4e1d-886c-9f2f986817df","Type":"ContainerStarted","Data":"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158"} Aug 04 23:14:00.933451 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:00.933098 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" event={"ID":"3099fcf6-f44d-4e1d-886c-9f2f986817df","Type":"ContainerStarted","Data":"2fbbb378289854ff56980f1ea8fc78e776f972c5539ca2a9268f5dbc07571f69"} Aug 04 23:14:00.964268 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:00.964219 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" podStartSLOduration=1.96420445 podStartE2EDuration="1.96420445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:14:00.96222698 +0000 UTC m=+3183.905733210" watchObservedRunningTime="2026-08-04 23:14:00.96420445 +0000 UTC m=+3183.907710681" Aug 04 23:14:09.809004 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:09.808970 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:14:09.809427 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:09.809021 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:14:09.821369 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:09.821344 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:14:09.980770 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:14:09.980744 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:15:10.490342 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:10.490265 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:15:25.509425 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.509371 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" podUID="3099fcf6-f44d-4e1d-886c-9f2f986817df" containerName="main" containerID="cri-o://3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158" gracePeriod=45 Aug 04 23:15:25.742968 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.742944 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:15:25.843174 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843092 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxcpl\" (UniqueName: \"kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843174 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843147 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843180 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843249 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843287 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843303 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir\") pod \"3099fcf6-f44d-4e1d-886c-9f2f986817df\" (UID: \"3099fcf6-f44d-4e1d-886c-9f2f986817df\") " Aug 04 23:15:25.843589 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843415 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache" (OuterVolumeSpecName: "model-cache") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:15:25.843646 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843583 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home" (OuterVolumeSpecName: "home") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:15:25.843646 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843592 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:15:25.843646 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.843599 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-model-cache\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:25.845393 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.845364 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm" (OuterVolumeSpecName: "dshm") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:15:25.845502 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.845411 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 23:15:25.845502 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.845414 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl" (OuterVolumeSpecName: "kube-api-access-rxcpl") pod "3099fcf6-f44d-4e1d-886c-9f2f986817df" (UID: "3099fcf6-f44d-4e1d-886c-9f2f986817df"). InnerVolumeSpecName "kube-api-access-rxcpl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:15:25.944911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.944865 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-dshm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:25.944911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.944904 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3099fcf6-f44d-4e1d-886c-9f2f986817df-tls-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:25.944911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.944914 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-home\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:25.944911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.944922 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3099fcf6-f44d-4e1d-886c-9f2f986817df-tmp-dir\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:25.945204 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:25.944931 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxcpl\" (UniqueName: \"kubernetes.io/projected/3099fcf6-f44d-4e1d-886c-9f2f986817df-kube-api-access-rxcpl\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:15:26.273679 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.273638 2571 generic.go:356] "Generic (PLEG): container finished" podID="3099fcf6-f44d-4e1d-886c-9f2f986817df" containerID="3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158" exitCode=0 Aug 04 23:15:26.273841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.273685 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" Aug 04 23:15:26.273841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.273696 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" event={"ID":"3099fcf6-f44d-4e1d-886c-9f2f986817df","Type":"ContainerDied","Data":"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158"} Aug 04 23:15:26.273841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.273736 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t" event={"ID":"3099fcf6-f44d-4e1d-886c-9f2f986817df","Type":"ContainerDied","Data":"2fbbb378289854ff56980f1ea8fc78e776f972c5539ca2a9268f5dbc07571f69"} Aug 04 23:15:26.273841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.273758 2571 scope.go:117] "RemoveContainer" containerID="3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158" Aug 04 23:15:26.282560 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.282540 2571 scope.go:117] "RemoveContainer" containerID="3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158" Aug 04 23:15:26.282813 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:15:26.282793 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158\": container with ID starting with 3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158 not found: ID does not exist" containerID="3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158" Aug 04 23:15:26.282879 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.282821 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158"} err="failed to get container status \"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158\": rpc error: code = NotFound desc = could not find container \"3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158\": container with ID starting with 3351ecfe1c2ef0b2615ed1a7a4f75c2b65bb54f555e4bf603071d66cb8d91158 not found: ID does not exist" Aug 04 23:15:26.293798 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.293775 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:15:26.296724 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:26.296702 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-55909a/sub-test-model-55909a-kserve-6f8c8fc685-qwv2t"] Aug 04 23:15:27.601739 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:27.601710 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3099fcf6-f44d-4e1d-886c-9f2f986817df" path="/var/lib/kubelet/pods/3099fcf6-f44d-4e1d-886c-9f2f986817df/volumes" Aug 04 23:15:30.897020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.896986 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:15:30.897399 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.897278 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3099fcf6-f44d-4e1d-886c-9f2f986817df" containerName="main" Aug 04 23:15:30.897399 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.897289 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3099fcf6-f44d-4e1d-886c-9f2f986817df" containerName="main" Aug 04 23:15:30.897399 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.897336 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3099fcf6-f44d-4e1d-886c-9f2f986817df" containerName="main" Aug 04 23:15:30.901757 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.901739 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.904166 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.904143 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"openshift-service-ca.crt\"" Aug 04 23:15:30.905176 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.905159 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"rate-test-model-ec06a8-kserve-self-signed-certs\"" Aug 04 23:15:30.905274 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.905185 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"kube-root-ca.crt\"" Aug 04 23:15:30.905274 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.905163 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-ec06a8\"/\"default-dockercfg-g4dlq\"" Aug 04 23:15:30.910272 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.910248 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:15:30.985862 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.985831 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.986058 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.985881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.986058 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.985909 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxw84\" (UniqueName: \"kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.986058 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.985944 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.986058 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.985964 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:30.986242 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:30.986095 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087149 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087119 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087149 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087150 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087370 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087169 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxw84\" (UniqueName: \"kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087370 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087291 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087370 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087325 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087370 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087350 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087586 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087567 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087687 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087662 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.087687 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.087681 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.089535 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.089510 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.089761 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.089745 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.096652 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.096626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxw84\" (UniqueName: \"kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84\") pod \"rate-test-model-ec06a8-kserve-5cdf9fc999-j488t\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.211240 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.211213 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:31.336164 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:31.336103 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:15:31.338362 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:15:31.338333 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbf6caa8_a3f8_479d_82bc_d6224efcdaf8.slice/crio-345ab1d913e624491f0a20a551d800d755d2777c79d7659a84b7b1d22324cef2 WatchSource:0}: Error finding container 345ab1d913e624491f0a20a551d800d755d2777c79d7659a84b7b1d22324cef2: Status 404 returned error can't find the container with id 345ab1d913e624491f0a20a551d800d755d2777c79d7659a84b7b1d22324cef2 Aug 04 23:15:32.299720 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:32.299681 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" event={"ID":"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8","Type":"ContainerStarted","Data":"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44"} Aug 04 23:15:32.299720 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:32.299723 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" event={"ID":"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8","Type":"ContainerStarted","Data":"345ab1d913e624491f0a20a551d800d755d2777c79d7659a84b7b1d22324cef2"} Aug 04 23:15:32.316372 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:32.316327 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" podStartSLOduration=2.316312425 podStartE2EDuration="2.316312425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:15:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:15:32.314806038 +0000 UTC m=+3275.258312266" watchObservedRunningTime="2026-08-04 23:15:32.316312425 +0000 UTC m=+3275.259818655" Aug 04 23:15:41.212098 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:41.212061 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:41.212098 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:41.212110 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:41.224657 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:41.224635 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:15:41.352224 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:15:41.352200 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:17:38.798597 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:38.798567 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:17:53.817279 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:53.817170 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" podUID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" containerName="main" containerID="cri-o://1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44" gracePeriod=45 Aug 04 23:17:54.058962 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.058936 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:17:54.147815 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147739 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.147815 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147799 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.148018 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147833 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.148018 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147879 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.148018 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147901 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.148018 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.147968 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxw84\" (UniqueName: \"kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84\") pod \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\" (UID: \"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8\") " Aug 04 23:17:54.148236 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.148144 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home" (OuterVolumeSpecName: "home") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:17:54.148236 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.148156 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache" (OuterVolumeSpecName: "model-cache") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:17:54.148330 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.148234 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:17:54.148330 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.148261 2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-home\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.148330 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.148280 2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-model-cache\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.150029 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.150006 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84" (OuterVolumeSpecName: "kube-api-access-sxw84") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "kube-api-access-sxw84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:17:54.150368 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.150349 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm" (OuterVolumeSpecName: "dshm") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:17:54.150412 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.150367 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" (UID: "fbf6caa8-a3f8-479d-82bc-d6224efcdaf8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 23:17:54.248963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.248923 2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tmp-dir\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.248963 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.248959 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxw84\" (UniqueName: \"kubernetes.io/projected/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-kube-api-access-sxw84\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.249149 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.248972 2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-tls-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.249149 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.248985 2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8-dshm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:17:54.894219 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.894180 2571 generic.go:356] "Generic (PLEG): container finished" podID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" containerID="1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44" exitCode=0 Aug 04 23:17:54.894779 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.894240 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" Aug 04 23:17:54.894779 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.894262 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" event={"ID":"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8","Type":"ContainerDied","Data":"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44"} Aug 04 23:17:54.894779 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.894309 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t" event={"ID":"fbf6caa8-a3f8-479d-82bc-d6224efcdaf8","Type":"ContainerDied","Data":"345ab1d913e624491f0a20a551d800d755d2777c79d7659a84b7b1d22324cef2"} Aug 04 23:17:54.894779 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.894328 2571 scope.go:117] "RemoveContainer" containerID="1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44" Aug 04 23:17:54.903821 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.903799 2571 scope.go:117] "RemoveContainer" containerID="1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44" Aug 04 23:17:54.904121 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:17:54.904096 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44\": container with ID starting with 1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44 not found: ID does not exist" containerID="1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44" Aug 04 23:17:54.904197 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.904134 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44"} err="failed to get container status \"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44\": rpc error: code = NotFound desc = could not find container \"1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44\": container with ID starting with 1cb66a8038b81fc8eca46bf3a14dc065a24d6c53108968c95183a735d592cb44 not found: ID does not exist" Aug 04 23:17:54.913973 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.913949 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:17:54.919628 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:54.919609 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-ec06a8/rate-test-model-ec06a8-kserve-5cdf9fc999-j488t"] Aug 04 23:17:55.602468 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:55.602438 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" path="/var/lib/kubelet/pods/fbf6caa8-a3f8-479d-82bc-d6224efcdaf8/volumes" Aug 04 23:17:59.329126 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.329092 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:17:59.329486 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.329391 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" containerName="main" Aug 04 23:17:59.329486 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.329403 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" containerName="main" Aug 04 23:17:59.329486 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.329461 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf6caa8-a3f8-479d-82bc-d6224efcdaf8" containerName="main" Aug 04 23:17:59.333721 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.333702 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.335785 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.335767 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-d1b011-openshift-default-dockercfg-8hkmf\"" Aug 04 23:17:59.343202 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.343179 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:17:59.391529 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391505 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391639 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391544 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391639 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391564 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391639 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391585 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391669 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391710 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391743 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391779 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vnvl\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.391864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.391809 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.492781 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.492753 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vnvl\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.492900 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.492786 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.492900 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.492821 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.492900 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.492876 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493090 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.492905 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493090 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493027 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493204 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493126 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493204 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493151 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493204 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493175 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493381 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493360 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493440 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493401 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493535 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493513 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493594 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493566 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.493653 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.493634 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.495309 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.495286 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.495495 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.495479 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.499909 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.499887 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.499985 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.499888 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vnvl\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl\") pod \"e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.643663 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.643582 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:17:59.769960 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.769916 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:17:59.772844 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:17:59.772821 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc487a23d_8ede_4bfd_b062_df321d758d17.slice/crio-b7f0839a3b9c5fbdd58f28bccc99a9732b841af456b64e080b6d2cb4f1f19e77 WatchSource:0}: Error finding container b7f0839a3b9c5fbdd58f28bccc99a9732b841af456b64e080b6d2cb4f1f19e77: Status 404 returned error can't find the container with id b7f0839a3b9c5fbdd58f28bccc99a9732b841af456b64e080b6d2cb4f1f19e77 Aug 04 23:17:59.775365 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.775332 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:17:59.775461 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.775395 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:17:59.775461 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.775421 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:17:59.917277 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.917195 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerStarted","Data":"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94"} Aug 04 23:17:59.917277 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.917261 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerStarted","Data":"b7f0839a3b9c5fbdd58f28bccc99a9732b841af456b64e080b6d2cb4f1f19e77"} Aug 04 23:17:59.938309 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:17:59.938253 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podStartSLOduration=0.938236537 podStartE2EDuration="938.236537ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:17:59.937977175 +0000 UTC m=+3422.881483408" watchObservedRunningTime="2026-08-04 23:17:59.938236537 +0000 UTC m=+3422.881742814" Aug 04 23:18:00.644318 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:00.644289 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:00.649233 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:00.649208 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:00.920767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:00.920690 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:00.921527 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:00.921510 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:15.921904 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.921854 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:15.922356 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.921942 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:15.995118 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.995083 2571 generic.go:356] "Generic (PLEG): container finished" podID="c487a23d-8ede-4bfd-b062-df321d758d17" containerID="e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94" exitCode=0 Aug 04 23:18:15.995267 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.995158 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerDied","Data":"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94"} Aug 04 23:18:15.995523 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.995509 2571 scope.go:117] "RemoveContainer" containerID="e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94" Aug 04 23:18:15.996458 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.996433 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:15.996512 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.996499 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:15.996547 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:15.996532 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:17.001364 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:17.001329 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerStarted","Data":"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6"} Aug 04 23:18:17.644121 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:17.644087 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:17.644422 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:17.644200 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:17.644603 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:17.644571 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:18.644497 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:18.644460 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:18.645095 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:18.645059 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:19.644409 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:19.644374 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:19.644703 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:19.644676 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:20.644911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:20.644874 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:20.645488 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:20.645455 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:21.644434 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:21.644390 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Aug 04 23:18:21.644643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:21.644455 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Aug 04 23:18:22.648308 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:22.648275 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:22.648696 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:22.648543 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:22.649121 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:22.649104 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:18:27.814854 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.814824 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:18:27.817069 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.817029 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.819280 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.819217 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-edf378-openshift-default-dockercfg-6c2x2\"" Aug 04 23:18:27.830827 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.830805 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:18:27.945461 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945412 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945620 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945470 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945620 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945555 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945620 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945600 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945725 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945667 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945725 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945690 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945725 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945820 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945738 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr7tm\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:27.945820 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:27.945774 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046567 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046536 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046577 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046625 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046645 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046661 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046681 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr7tm\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.046738 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046718 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047054 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.046775 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047119 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047089 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047172 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047124 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047226 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047183 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047226 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047191 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047511 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047490 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.047565 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.047490 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.049407 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.049385 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.049637 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.049620 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.054078 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.054052 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.054308 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.054288 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr7tm\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm\") pod \"e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.127587 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.127533 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:28.251114 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.251088 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:18:28.253260 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:18:28.253229 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e4ead0_0f3a_452b_91e8_4d61fd0dbc6f.slice/crio-53cd4321893710a7e9c3b9d75380afa7f9a4ab21e4d9c640ed1bbb54e875dd3d WatchSource:0}: Error finding container 53cd4321893710a7e9c3b9d75380afa7f9a4ab21e4d9c640ed1bbb54e875dd3d: Status 404 returned error can't find the container with id 53cd4321893710a7e9c3b9d75380afa7f9a4ab21e4d9c640ed1bbb54e875dd3d Aug 04 23:18:28.257850 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.257812 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:28.257985 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.257889 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:28.257985 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:28.257939 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:29.049366 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:29.049328 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerStarted","Data":"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085"} Aug 04 23:18:29.049831 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:29.049426 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerStarted","Data":"53cd4321893710a7e9c3b9d75380afa7f9a4ab21e4d9c640ed1bbb54e875dd3d"} Aug 04 23:18:29.067662 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:29.067598 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podStartSLOduration=2.067584091 podStartE2EDuration="2.067584091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:18:29.067068773 +0000 UTC m=+3452.010575005" watchObservedRunningTime="2026-08-04 23:18:29.067584091 +0000 UTC m=+3452.011090321" Aug 04 23:18:29.128322 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:29.128295 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:29.133389 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:29.133367 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:30.054499 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:30.054466 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:30.055509 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:30.055487 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:45.429226 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:45.429185 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:50658->10.133.0.33:15021: read: connection reset by peer" start-of-body= Aug 04 23:18:45.429654 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:45.429273 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": read tcp 10.133.0.2:50658->10.133.0.33:15021: read: connection reset by peer" Aug 04 23:18:46.125237 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.125204 2571 generic.go:356] "Generic (PLEG): container finished" podID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerID="0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085" exitCode=0 Aug 04 23:18:46.125417 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.125279 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerDied","Data":"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085"} Aug 04 23:18:46.125687 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.125674 2571 scope.go:117] "RemoveContainer" containerID="0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085" Aug 04 23:18:46.126767 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.126726 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:46.126831 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.126811 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:46.126880 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:46.126844 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:18:47.132275 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:47.132238 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerStarted","Data":"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2"} Aug 04 23:18:48.127925 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:48.127892 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:48.128150 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:48.128128 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Aug 04 23:18:48.128204 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:48.128177 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Aug 04 23:18:49.128536 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:49.128451 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Aug 04 23:18:49.128536 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:49.128501 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Aug 04 23:18:50.127928 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:50.127887 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Aug 04 23:18:50.128135 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:50.127951 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Aug 04 23:18:51.128203 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:51.128167 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Aug 04 23:18:51.128600 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:51.128224 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Aug 04 23:18:52.132908 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:52.132874 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:52.133315 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:52.133263 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:18:52.133974 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:18:52.133951 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:20:14.324911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.324878 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:20:14.327438 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.327418 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.329682 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.329661 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-2b5cc7-openshift-default-dockercfg-5d248\"" Aug 04 23:20:14.339948 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.339907 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:20:14.430444 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430420 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430577 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430461 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz2nh\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430577 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430484 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430577 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430522 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430577 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430547 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430711 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430595 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430711 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430624 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430711 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430688 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.430805 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.430729 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531336 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531304 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531359 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531387 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531418 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz2nh\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531439 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531466 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531490 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531540 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531538 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531877 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531584 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531877 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531836 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531987 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531945 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.531987 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.531963 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.532228 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.532203 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.532353 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.532275 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.533841 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.533822 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.533967 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.533950 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.538434 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.538409 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.538723 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.538707 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz2nh\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh\") pod \"e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.638081 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.638015 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:14.762391 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.762324 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:20:14.765111 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:20:14.765082 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5223afa_2a87_4fd4_b413_99a3b57e7ef5.slice/crio-f1964f83458046cd8a1bdb9040844f66dbefc3f84f1928cf14f65f227b309f6d WatchSource:0}: Error finding container f1964f83458046cd8a1bdb9040844f66dbefc3f84f1928cf14f65f227b309f6d: Status 404 returned error can't find the container with id f1964f83458046cd8a1bdb9040844f66dbefc3f84f1928cf14f65f227b309f6d Aug 04 23:20:14.767342 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.767313 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:14.767401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.767374 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:14.767445 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:14.767403 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:15.504869 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:15.504830 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerStarted","Data":"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5"} Aug 04 23:20:15.505274 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:15.504879 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerStarted","Data":"f1964f83458046cd8a1bdb9040844f66dbefc3f84f1928cf14f65f227b309f6d"} Aug 04 23:20:15.522839 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:15.522793 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podStartSLOduration=1.522778426 podStartE2EDuration="1.522778426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 23:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:20:15.521608629 +0000 UTC m=+3558.465114859" watchObservedRunningTime="2026-08-04 23:20:15.522778426 +0000 UTC m=+3558.466284657" Aug 04 23:20:15.638769 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:15.638491 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:15.645715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:15.645683 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:16.509288 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:16.509259 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:16.510386 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:16.510359 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:32.551237 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:32.551183 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 23:20:32.551648 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:32.551274 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 23:20:34.593084 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.592993 2571 generic.go:356] "Generic (PLEG): container finished" podID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerID="63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5" exitCode=0 Aug 04 23:20:34.593445 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.593072 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerDied","Data":"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5"} Aug 04 23:20:34.593522 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.593501 2571 scope.go:117] "RemoveContainer" containerID="63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5" Aug 04 23:20:34.594571 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.594539 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:34.594716 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.594602 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:34.594716 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:34.594635 2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Aug 04 23:20:35.600732 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:35.600699 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerStarted","Data":"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6"} Aug 04 23:20:35.639179 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:35.639155 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:35.639380 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:35.639359 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:35.639433 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:35.639402 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:36.639002 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:36.638967 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:36.639462 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:36.639031 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:37.639127 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:37.639097 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:37.639496 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:37.639149 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:38.638417 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:38.638384 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:38.638592 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:38.638432 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:39.638545 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:39.638504 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:39.638911 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:39.638567 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:40.638921 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:40.638889 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Aug 04 23:20:40.639324 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:40.638940 2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Aug 04 23:20:41.642134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:41.642106 2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:41.642496 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:41.642390 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:41.643132 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:41.643115 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:20:55.601798 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:55.601771 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:20:55.602218 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:55.601977 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" containerID="cri-o://f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6" gracePeriod=30 Aug 04 23:20:56.645283 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:56.645246 2571 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 23:20:56.645656 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:20:56.645302 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 23:21:00.842712 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.842685 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:21:00.925318 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925241 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925318 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925281 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925318 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925318 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925353 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925370 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925419 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925453 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925492 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925496 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:21:00.925572 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925524 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz2nh\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh\") pod \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\" (UID: \"a5223afa-2a87-4fd4-b413-99a3b57e7ef5\") " Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925661 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925718 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925809 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925883 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925912 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925925 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925936 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:00.925943 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.925935 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:21:00.927509 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.927483 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:21:00.927631 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.927556 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:21:00.927631 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.927596 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:21:00.927733 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:00.927708 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh" (OuterVolumeSpecName: "kube-api-access-bz2nh") pod "a5223afa-2a87-4fd4-b413-99a3b57e7ef5" (UID: "a5223afa-2a87-4fd4-b413-99a3b57e7ef5"). InnerVolumeSpecName "kube-api-access-bz2nh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:21:01.027256 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.027228 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:01.027256 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.027251 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz2nh\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-kube-api-access-bz2nh\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:01.027256 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.027261 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:01.027429 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.027269 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:01.027429 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.027277 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5223afa-2a87-4fd4-b413-99a3b57e7ef5-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:21:01.708794 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.708761 2571 generic.go:356] "Generic (PLEG): container finished" podID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerID="f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6" exitCode=0 Aug 04 23:21:01.708944 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.708803 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" Aug 04 23:21:01.708944 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.708806 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerDied","Data":"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6"} Aug 04 23:21:01.708944 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.708853 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4" event={"ID":"a5223afa-2a87-4fd4-b413-99a3b57e7ef5","Type":"ContainerDied","Data":"f1964f83458046cd8a1bdb9040844f66dbefc3f84f1928cf14f65f227b309f6d"} Aug 04 23:21:01.708944 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.708875 2571 scope.go:117] "RemoveContainer" containerID="f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6" Aug 04 23:21:01.717217 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.717204 2571 scope.go:117] "RemoveContainer" containerID="63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5" Aug 04 23:21:01.725400 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.725383 2571 scope.go:117] "RemoveContainer" containerID="f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6" Aug 04 23:21:01.725644 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:21:01.725627 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6\": container with ID starting with f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6 not found: ID does not exist" containerID="f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6" Aug 04 23:21:01.725708 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.725652 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6"} err="failed to get container status \"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6\": rpc error: code = NotFound desc = could not find container \"f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6\": container with ID starting with f91438ec5a05f765edc5752310a11dba8ad3fea283031ce5b0548ecf74f7c3c6 not found: ID does not exist" Aug 04 23:21:01.725708 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.725664 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:21:01.725708 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.725672 2571 scope.go:117] "RemoveContainer" containerID="63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5" Aug 04 23:21:01.725917 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:21:01.725897 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5\": container with ID starting with 63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5 not found: ID does not exist" containerID="63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5" Aug 04 23:21:01.725958 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.725925 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5"} err="failed to get container status \"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5\": rpc error: code = NotFound desc = could not find container \"63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5\": container with ID starting with 63ec6944fc550c370e7c8529e5160bb12719e98c8b4d2ae3cd890481fc252be5 not found: ID does not exist" Aug 04 23:21:01.729480 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:01.729454 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2b5cc7-openshift-default-76b7ddb46d-bwzg4"] Aug 04 23:21:03.600503 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:21:03.600460 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" path="/var/lib/kubelet/pods/a5223afa-2a87-4fd4-b413-99a3b57e7ef5/volumes" Aug 04 23:22:03.114965 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:03.114927 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:22:03.115415 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:03.115202 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" containerID="cri-o://51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6" gracePeriod=30 Aug 04 23:22:07.651241 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:07.651204 2571 patch_prober.go:28] interesting pod/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 23:22:07.651641 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:07.651265 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 23:22:08.359273 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.359249 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:22:08.483999 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.483972 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484018 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484058 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484082 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vnvl\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484108 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484129 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484190 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484161 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484211 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484238 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket\") pod \"c487a23d-8ede-4bfd-b062-df321d758d17\" (UID: \"c487a23d-8ede-4bfd-b062-df321d758d17\") " Aug 04 23:22:08.484508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484428 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:08.484508 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484472 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data" (OuterVolumeSpecName: "istio-data") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:08.484715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484505 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:22:08.484715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484578 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.484715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484627 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.484715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484632 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:08.484715 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.484645 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:08.486460 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.486433 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:08.486460 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.486449 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl" (OuterVolumeSpecName: "kube-api-access-8vnvl") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "kube-api-access-8vnvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:22:08.486596 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.486458 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token" (OuterVolumeSpecName: "istio-token") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:22:08.486596 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.486455 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c487a23d-8ede-4bfd-b062-df321d758d17" (UID: "c487a23d-8ede-4bfd-b062-df321d758d17"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:22:08.586019 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.585983 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vnvl\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-kube-api-access-8vnvl\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586019 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586013 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586019 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586023 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c487a23d-8ede-4bfd-b062-df321d758d17-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586250 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586031 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586250 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586061 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c487a23d-8ede-4bfd-b062-df321d758d17-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586250 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586070 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c487a23d-8ede-4bfd-b062-df321d758d17-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.586250 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.586077 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c487a23d-8ede-4bfd-b062-df321d758d17-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:08.988914 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.988878 2571 generic.go:356] "Generic (PLEG): container finished" podID="c487a23d-8ede-4bfd-b062-df321d758d17" containerID="51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6" exitCode=0 Aug 04 23:22:08.989290 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.988924 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" Aug 04 23:22:08.989290 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.988957 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerDied","Data":"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6"} Aug 04 23:22:08.989290 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.989010 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d" event={"ID":"c487a23d-8ede-4bfd-b062-df321d758d17","Type":"ContainerDied","Data":"b7f0839a3b9c5fbdd58f28bccc99a9732b841af456b64e080b6d2cb4f1f19e77"} Aug 04 23:22:08.989290 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:08.989031 2571 scope.go:117] "RemoveContainer" containerID="51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6" Aug 04 23:22:09.000075 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.000057 2571 scope.go:117] "RemoveContainer" containerID="e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94" Aug 04 23:22:09.007445 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.007427 2571 scope.go:117] "RemoveContainer" containerID="51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6" Aug 04 23:22:09.007686 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:22:09.007667 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6\": container with ID starting with 51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6 not found: ID does not exist" containerID="51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6" Aug 04 23:22:09.007732 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.007695 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6"} err="failed to get container status \"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6\": rpc error: code = NotFound desc = could not find container \"51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6\": container with ID starting with 51292cffa09d38ddfcf7cbce67bb9a2a3b73855955484aea839848714852ece6 not found: ID does not exist" Aug 04 23:22:09.007732 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.007713 2571 scope.go:117] "RemoveContainer" containerID="e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94" Aug 04 23:22:09.007946 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:22:09.007930 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94\": container with ID starting with e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94 not found: ID does not exist" containerID="e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94" Aug 04 23:22:09.007997 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.007948 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94"} err="failed to get container status \"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94\": rpc error: code = NotFound desc = could not find container \"e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94\": container with ID starting with e977e32973b7d86376be74d35d3db63ef51f492e04ad07cb4fc71fb7e18bff94 not found: ID does not exist" Aug 04 23:22:09.011940 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.011918 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:22:09.014922 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.014903 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-d1b011-openshift-default-54677bcb6d-qqn7d"] Aug 04 23:22:09.602117 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:09.602078 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" path="/var/lib/kubelet/pods/c487a23d-8ede-4bfd-b062-df321d758d17/volumes" Aug 04 23:22:20.549581 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:20.549489 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:22:20.550353 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:20.549735 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" containerID="cri-o://d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2" gracePeriod=30 Aug 04 23:22:22.135748 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:22.135708 2571 patch_prober.go:28] interesting pod/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 23:22:22.136142 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:22.135769 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 23:22:25.793169 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.793138 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:22:25.820359 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820288 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820359 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820317 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820359 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820339 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820359 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820358 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820388 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820453 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820510 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr7tm\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820536 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820567 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820611 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket\") pod \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\" (UID: \"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f\") " Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820640 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:25.820672 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820656 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:25.821151 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820836 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:22:25.821151 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820861 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.821151 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820880 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.821151 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820893 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.821151 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.820894 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:25.822762 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.822734 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm" (OuterVolumeSpecName: "kube-api-access-dr7tm") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "kube-api-access-dr7tm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:22:25.822762 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.822748 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:22:25.822920 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.822883 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:22:25.823007 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.822990 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" (UID: "f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:22:25.922198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922164 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr7tm\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-kube-api-access-dr7tm\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.922198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922193 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.922198 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922203 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.922406 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922213 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.922406 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922221 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:25.922406 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:25.922229 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:22:26.064851 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.064818 2571 generic.go:356] "Generic (PLEG): container finished" podID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerID="d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2" exitCode=0 Aug 04 23:22:26.065020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.064865 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" Aug 04 23:22:26.065020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.064875 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerDied","Data":"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2"} Aug 04 23:22:26.065020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.064916 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp" event={"ID":"f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f","Type":"ContainerDied","Data":"53cd4321893710a7e9c3b9d75380afa7f9a4ab21e4d9c640ed1bbb54e875dd3d"} Aug 04 23:22:26.065020 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.064937 2571 scope.go:117] "RemoveContainer" containerID="d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2" Aug 04 23:22:26.073649 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.073632 2571 scope.go:117] "RemoveContainer" containerID="0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085" Aug 04 23:22:26.081012 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.080994 2571 scope.go:117] "RemoveContainer" containerID="d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2" Aug 04 23:22:26.081249 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:22:26.081230 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2\": container with ID starting with d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2 not found: ID does not exist" containerID="d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2" Aug 04 23:22:26.081313 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.081255 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2"} err="failed to get container status \"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2\": rpc error: code = NotFound desc = could not find container \"d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2\": container with ID starting with d239f628b701d21f51f207aafe04d4ccbf515bd55ae6e3ccf4c2730e3bfbddf2 not found: ID does not exist" Aug 04 23:22:26.081313 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.081270 2571 scope.go:117] "RemoveContainer" containerID="0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085" Aug 04 23:22:26.081467 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:22:26.081452 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085\": container with ID starting with 0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085 not found: ID does not exist" containerID="0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085" Aug 04 23:22:26.081505 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.081469 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085"} err="failed to get container status \"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085\": rpc error: code = NotFound desc = could not find container \"0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085\": container with ID starting with 0a9e7f47edd73fc6a6a6b8930690ca34feece32c69c15f74e9f638cae5f1b085 not found: ID does not exist" Aug 04 23:22:26.084288 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.084266 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:22:26.086158 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:26.086141 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-edf378-openshift-default-69b97b9fc4-2l2gp"] Aug 04 23:22:27.600621 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:27.600587 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" path="/var/lib/kubelet/pods/f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f/volumes" Aug 04 23:22:55.237755 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:55.237721 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:22:55.238284 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:22:55.238018 2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" containerID="cri-o://b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c" gracePeriod=30 Aug 04 23:23:00.477549 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.477524 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:23:00.604771 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604697 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.604771 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604744 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.604946 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604820 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b78z4\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.604946 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604880 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.604946 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604911 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.605134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604948 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.605134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.604984 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.605134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605016 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.605134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605069 2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert\") pod \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\" (UID: \"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15\") " Aug 04 23:23:00.605134 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605120 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:23:00.605401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605169 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data" (OuterVolumeSpecName: "istio-data") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:23:00.605401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605271 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:23:00.605401 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605285 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:23:00.605569 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605418 2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.605569 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605440 2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-workload-certs\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.605569 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605454 2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-credential-socket\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.605569 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605456 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 23:23:00.605569 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.605466 2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-data\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.607074 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.607046 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4" (OuterVolumeSpecName: "kube-api-access-b78z4") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "kube-api-access-b78z4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:23:00.607202 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.607152 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 23:23:00.607488 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.607473 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 23:23:00.607537 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.607515 2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token" (OuterVolumeSpecName: "istio-token") pod "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" (UID: "9b3d4ed4-eac5-48f5-adff-399f3bdc2a15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 23:23:00.705777 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.705754 2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-envoy\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.705777 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.705775 2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-podinfo\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.705935 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.705784 2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istiod-ca-cert\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.705935 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.705792 2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-istio-token\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:00.705935 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:00.705799 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b78z4\" (UniqueName: \"kubernetes.io/projected/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15-kube-api-access-b78z4\") on node \"ip-10-0-137-148.ec2.internal\" DevicePath \"\"" Aug 04 23:23:01.209425 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.209390 2571 generic.go:356] "Generic (PLEG): container finished" podID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerID="b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c" exitCode=0 Aug 04 23:23:01.209601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.209426 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerDied","Data":"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c"} Aug 04 23:23:01.209601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.209451 2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" Aug 04 23:23:01.209601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.209493 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7" event={"ID":"9b3d4ed4-eac5-48f5-adff-399f3bdc2a15","Type":"ContainerDied","Data":"691cbc5a3bbe89f756f478606dc73d767a6f7bb692b5c2246e5c02e67415bb49"} Aug 04 23:23:01.209601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.209525 2571 scope.go:117] "RemoveContainer" containerID="b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c" Aug 04 23:23:01.218507 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.218490 2571 scope.go:117] "RemoveContainer" containerID="48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a" Aug 04 23:23:01.225607 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.225592 2571 scope.go:117] "RemoveContainer" containerID="b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c" Aug 04 23:23:01.225844 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:23:01.225828 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c\": container with ID starting with b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c not found: ID does not exist" containerID="b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c" Aug 04 23:23:01.225890 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.225850 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c"} err="failed to get container status \"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c\": rpc error: code = NotFound desc = could not find container \"b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c\": container with ID starting with b3a974ceff2a541ce01d992c44193f33765ea51ca8ea98c278dd6f13adccb68c not found: ID does not exist" Aug 04 23:23:01.225890 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.225866 2571 scope.go:117] "RemoveContainer" containerID="48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a" Aug 04 23:23:01.226075 ip-10-0-137-148 kubenswrapper[2571]: E0804 23:23:01.226059 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a\": container with ID starting with 48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a not found: ID does not exist" containerID="48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a" Aug 04 23:23:01.226123 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.226079 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a"} err="failed to get container status \"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a\": rpc error: code = NotFound desc = could not find container \"48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a\": container with ID starting with 48f1f6549ea3766f81e2cebf3603b3f5d61f0ccf753610bd3d0b28c948fb8e7a not found: ID does not exist" Aug 04 23:23:01.229492 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.229470 2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:23:01.232562 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.232541 2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-55909a-openshift-default-55b5bd5d9b-tr8c7"] Aug 04 23:23:01.601279 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.601200 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" path="/var/lib/kubelet/pods/9b3d4ed4-eac5-48f5-adff-399f3bdc2a15/volumes" Aug 04 23:23:01.758156 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.758130 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6cdbbfdb-s2659_a36948aa-f972-4f29-9a76-bad94800429a/copy-manifests/0.log" Aug 04 23:23:01.764389 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:01.764352 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6cdbbfdb-s2659_a36948aa-f972-4f29-9a76-bad94800429a/manager/0.log" Aug 04 23:23:02.639261 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:02.639230 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hzm47_ff32be7b-f411-430c-be28-69fd484fd046/postgres/0.log" Aug 04 23:23:04.590880 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:04.590843 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ppc6v_86020cde-3513-4b1b-b491-b69292182c66/discovery/0.log" Aug 04 23:23:04.916561 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:04.916483 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/1.log" Aug 04 23:23:05.733917 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:05.733884 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-g4ckt_22b1c7cb-4142-44cf-86b9-87d7c920b6a0/main/0.log" Aug 04 23:23:05.963449 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:05.963415 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fp4pcj_6b787a9a-ee4e-4173-9e1f-c1f867c9fb5f/main/0.log" Aug 04 23:23:06.198389 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:06.198304 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-j89hs_aabc8086-1c11-4c3b-a675-916c15f36ef4/main/0.log" Aug 04 23:23:14.161094 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:14.161064 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d6wxm_1d994125-f976-49cd-aff5-f27c9b651d61/global-pull-secret-syncer/0.log" Aug 04 23:23:14.319824 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:14.319793 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-pdlln_58b2f6fe-ddca-4893-bf23-a2c844c8e419/konnectivity-agent/0.log" Aug 04 23:23:14.364384 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:14.364353 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-148.ec2.internal_697f105faa10eb4a5de45deb313e7223/haproxy/0.log" Aug 04 23:23:21.583454 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:21.583415 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-58754c9c68-m54z8_14c63d5d-571e-4485-beed-da671a98066c/metrics-server/0.log" Aug 04 23:23:21.641338 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:21.641310 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/node-exporter/0.log" Aug 04 23:23:21.660680 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:21.660651 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/kube-rbac-proxy/0.log" Aug 04 23:23:21.686891 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:21.686873 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-759bp_d2e56019-3d66-4892-bb07-927365b8cb05/init-textfile/0.log" Aug 04 23:23:24.209288 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209254 2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj"] Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209515 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209525 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209537 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209543 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209550 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209556 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209563 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209568 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209576 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209582 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209595 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209600 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209642 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" Aug 04 23:23:24.209643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209648 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e4ead0-0f3a-452b-91e8-4d61fd0dbc6f" containerName="istio-proxy" Aug 04 23:23:24.210083 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209655 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b3d4ed4-eac5-48f5-adff-399f3bdc2a15" containerName="istio-proxy" Aug 04 23:23:24.210083 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209662 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" Aug 04 23:23:24.210083 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209668 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c487a23d-8ede-4bfd-b062-df321d758d17" containerName="istio-proxy" Aug 04 23:23:24.210083 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209674 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.210083 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.209680 2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5223afa-2a87-4fd4-b413-99a3b57e7ef5" containerName="istio-proxy" Aug 04 23:23:24.212505 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.212485 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.214457 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.214439 2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6vkp5\"/\"default-dockercfg-grvmj\"" Aug 04 23:23:24.214551 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.214468 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vkp5\"/\"openshift-service-ca.crt\"" Aug 04 23:23:24.215233 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.215219 2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6vkp5\"/\"kube-root-ca.crt\"" Aug 04 23:23:24.221736 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.221717 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj"] Aug 04 23:23:24.282411 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.282379 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-proc\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.282527 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.282429 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb82d\" (UniqueName: \"kubernetes.io/projected/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-kube-api-access-nb82d\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.282527 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.282504 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-podres\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.282527 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.282523 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-sys\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.282643 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.282543 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-lib-modules\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383465 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383437 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-proc\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383485 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb82d\" (UniqueName: \"kubernetes.io/projected/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-kube-api-access-nb82d\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383527 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-podres\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383545 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-sys\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383563 2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-lib-modules\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383601 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383566 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-proc\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383832 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383626 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-sys\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383832 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383664 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-podres\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.383832 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.383763 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-lib-modules\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.390393 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.390374 2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb82d\" (UniqueName: \"kubernetes.io/projected/a49beb3a-2f85-44c5-8616-cfd1bdc81dd1-kube-api-access-nb82d\") pod \"perf-node-gather-daemonset-955dj\" (UID: \"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1\") " pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.522369 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.522304 2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:24.644348 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:24.644325 2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj"] Aug 04 23:23:24.646889 ip-10-0-137-148 kubenswrapper[2571]: W0804 23:23:24.646864 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda49beb3a_2f85_44c5_8616_cfd1bdc81dd1.slice/crio-73baa50db74dd2d63030dfc89bf06a7a645fff0448354e1c92e8df4040070deb WatchSource:0}: Error finding container 73baa50db74dd2d63030dfc89bf06a7a645fff0448354e1c92e8df4040070deb: Status 404 returned error can't find the container with id 73baa50db74dd2d63030dfc89bf06a7a645fff0448354e1c92e8df4040070deb Aug 04 23:23:25.307110 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:25.307075 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" event={"ID":"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1","Type":"ContainerStarted","Data":"f29a231490b5090888e852a28fd201403dd0f43815740a5fb9fc3880f62fb08f"} Aug 04 23:23:25.307518 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:25.307132 2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" event={"ID":"a49beb3a-2f85-44c5-8616-cfd1bdc81dd1","Type":"ContainerStarted","Data":"73baa50db74dd2d63030dfc89bf06a7a645fff0448354e1c92e8df4040070deb"} Aug 04 23:23:25.307518 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:25.307170 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:25.322864 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:25.322819 2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" podStartSLOduration=1.322804204 podStartE2EDuration="1.322804204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 23:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 23:23:25.321604037 +0000 UTC m=+3748.265110269" watchObservedRunningTime="2026-08-04 23:23:25.322804204 +0000 UTC m=+3748.266310434" Aug 04 23:23:26.852873 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:26.852840 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j2zms_df6f5703-7447-44a9-9fb4-fcc924d73400/dns/0.log" Aug 04 23:23:26.872744 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:26.872726 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-j2zms_df6f5703-7447-44a9-9fb4-fcc924d73400/kube-rbac-proxy/0.log" Aug 04 23:23:26.894659 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:26.894639 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m2wkk_340fd968-a788-4c39-b126-2cc4b93f4ffc/dns-node-resolver/0.log" Aug 04 23:23:27.470498 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:27.470455 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dcbff56cb-7jlgb_b19748d2-e482-4ad6-9e8e-8aaa463e0c20/registry/0.log" Aug 04 23:23:27.515308 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:27.515282 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wpdwv_e7a049d9-4840-4128-a132-b445eb958f08/node-ca/0.log" Aug 04 23:23:28.762127 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:28.762095 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ppc6v_86020cde-3513-4b1b-b491-b69292182c66/discovery/0.log" Aug 04 23:23:28.833557 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:28.833531 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/0.log" Aug 04 23:23:28.846186 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:28.846162 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/1.log" Aug 04 23:23:29.767797 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:29.767769 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wc957_efda0d95-7b75-435c-a1a9-e0eadf43d989/serve-healthcheck-canary/0.log" Aug 04 23:23:30.411874 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:30.411838 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pnlc9_1fa036cc-7ce3-4b4c-992d-ff5982e62b06/kube-rbac-proxy/0.log" Aug 04 23:23:30.431825 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:30.431801 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pnlc9_1fa036cc-7ce3-4b4c-992d-ff5982e62b06/exporter/0.log" Aug 04 23:23:30.453058 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:30.453021 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pnlc9_1fa036cc-7ce3-4b4c-992d-ff5982e62b06/extractor/0.log" Aug 04 23:23:31.320651 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:31.320622 2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6vkp5/perf-node-gather-daemonset-955dj" Aug 04 23:23:33.446393 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:33.446360 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6cdbbfdb-s2659_a36948aa-f972-4f29-9a76-bad94800429a/manager/0.log" Aug 04 23:23:33.467745 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:33.467722 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f6cdbbfdb-s2659_a36948aa-f972-4f29-9a76-bad94800429a/copy-manifests/0.log" Aug 04 23:23:34.919173 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:34.919144 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76f49b9fdf-wbssg_79b5d157-61d0-4351-a561-3dfcb01fe7f8/manager/0.log" Aug 04 23:23:35.550277 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:35.550196 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ppc6v_86020cde-3513-4b1b-b491-b69292182c66/discovery/0.log" Aug 04 23:23:35.599294 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:35.599259 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/0.log" Aug 04 23:23:35.616477 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:35.616450 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/1.log" Aug 04 23:23:37.352331 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:37.352288 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-ppc6v_86020cde-3513-4b1b-b491-b69292182c66/discovery/0.log" Aug 04 23:23:37.402917 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:37.402891 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/0.log" Aug 04 23:23:37.415342 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:37.415311 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5cmq9_312936f4-7905-4e61-bf78-46d9963e2b66/istio-proxy/1.log" Aug 04 23:23:41.177939 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.177910 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-56572_ca5398fd-a6b8-46ed-8891-aa72bf184169/kube-multus/0.log" Aug 04 23:23:41.200568 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.200546 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/kube-multus-additional-cni-plugins/0.log" Aug 04 23:23:41.218853 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.218835 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/egress-router-binary-copy/0.log" Aug 04 23:23:41.237116 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.237093 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/cni-plugins/0.log" Aug 04 23:23:41.256933 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.256914 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/bond-cni-plugin/0.log" Aug 04 23:23:41.276763 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.276744 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/routeoverride-cni/0.log" Aug 04 23:23:41.295188 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.295168 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/whereabouts-cni-bincopy/0.log" Aug 04 23:23:41.314404 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.314377 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2qkwg_32c59839-de1f-4afe-b061-3ab933711cd5/whereabouts-cni/0.log" Aug 04 23:23:41.738475 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.738451 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gzwl_f0dd7f79-567e-4af2-9787-647a72152f75/network-metrics-daemon/0.log" Aug 04 23:23:41.757202 ip-10-0-137-148 kubenswrapper[2571]: I0804 23:23:41.757176 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9gzwl_f0dd7f79-567e-4af2-9787-647a72152f75/kube-rbac-proxy/0.log"