Jul 22 04:31:01.347667 ip-10-0-133-75 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 04:31:01.347683 ip-10-0-133-75 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 04:31:01.347694 ip-10-0-133-75 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 04:31:01.348050 ip-10-0-133-75 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 04:31:11.450352 ip-10-0-133-75 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 04:31:11.450371 ip-10-0-133-75 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5b968bcc34bc4c9892c814f93e6cca60 -- Jul 22 04:33:04.847266 ip-10-0-133-75 systemd[1]: Starting Kubernetes Kubelet... Jul 22 04:33:05.218134 ip-10-0-133-75 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:33:05.218134 ip-10-0-133-75 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 04:33:05.218134 ip-10-0-133-75 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:33:05.218134 ip-10-0-133-75 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 04:33:05.218134 ip-10-0-133-75 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:33:05.220527 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.220444 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 04:33:05.225375 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225362 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.225375 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225373 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.225375 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225377 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225380 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225392 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225395 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225398 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225401 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225404 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225407 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225409 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225412 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225414 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225417 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225419 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225421 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225424 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225426 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225428 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225431 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225433 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225436 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.225501 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225438 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225441 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225443 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225445 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225448 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225450 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225453 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225455 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225458 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225460 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225462 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225465 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225467 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225470 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225473 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225476 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225478 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225481 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225483 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225486 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.226164 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225488 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225491 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225493 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225496 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225498 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225500 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225503 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225506 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225508 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225511 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225513 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225516 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225518 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225521 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225523 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225526 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225528 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225531 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225533 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225535 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.226835 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225538 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225540 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225543 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225546 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225548 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225551 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225555 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225557 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225560 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225562 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225564 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225567 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225569 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225573 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225577 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225579 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225582 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225585 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225587 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.227503 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225590 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225592 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225594 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225597 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225599 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225602 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225604 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225606 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225611 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225615 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225618 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225621 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225624 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225626 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225630 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225633 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225636 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.225639 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226281 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.228150 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226287 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226291 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226294 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226298 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226301 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226305 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226309 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226312 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226314 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226317 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226320 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226322 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226326 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226329 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226332 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226335 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226337 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226341 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226343 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.228765 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226346 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226348 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226351 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226353 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226356 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226359 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226361 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226364 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226367 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226369 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226372 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226374 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226377 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226379 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226381 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226384 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226387 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226389 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226392 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226394 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.229434 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226396 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226399 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226401 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226404 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226406 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226408 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226411 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226413 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226415 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226418 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226420 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226422 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226425 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226427 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226429 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226432 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226435 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226437 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226440 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226443 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.230138 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226445 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226447 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226450 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226453 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226455 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226457 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226460 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226462 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226465 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226467 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226470 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226472 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226475 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226477 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226479 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226481 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226484 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226486 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226488 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226491 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.230823 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226494 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226497 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226499 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226501 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226504 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226506 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226508 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226511 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226513 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226516 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226519 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226522 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226525 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226527 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226529 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226533 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226536 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226539 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.226542 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228614 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.231510 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228624 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228627 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228630 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228633 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228636 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228638 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228641 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228643 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228646 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228648 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228651 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228653 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228656 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228658 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228661 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228663 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228665 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228668 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228670 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228672 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.232230 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228675 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228678 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228680 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228682 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228685 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228687 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228691 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228694 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228698 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228702 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228705 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228717 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228720 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228724 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228726 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228729 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228732 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228735 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228737 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.232906 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228740 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228742 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228745 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228747 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228750 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228752 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228755 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228757 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228760 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228762 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228765 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228767 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228771 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228773 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228776 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228779 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228783 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228786 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228789 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.233555 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228791 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228794 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228797 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228799 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228802 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228805 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228808 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228811 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228813 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228816 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228818 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228821 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228823 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228826 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228829 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228831 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228834 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228836 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228838 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228841 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.234268 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228843 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228846 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228849 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228851 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228854 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228857 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228860 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228862 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228865 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228867 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228870 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228872 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228875 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228877 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228880 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228882 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228884 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228887 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228890 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.228893 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.234954 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228965 2555 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228980 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228987 2555 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228991 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228994 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.228997 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229001 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229005 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229008 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229010 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229014 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229017 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229033 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229036 2555 flags.go:64] FLAG: --cgroup-root="" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229039 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229042 2555 flags.go:64] FLAG: --client-ca-file="" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229044 2555 flags.go:64] FLAG: --cloud-provider="external" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229048 2555 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229052 2555 flags.go:64] FLAG: --cluster-domain="" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229054 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229057 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229060 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229063 2555 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229066 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 04:33:05.235638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229069 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229072 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229075 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229078 2555 flags.go:64] FLAG: --contention-profiling="false" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229081 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229084 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229089 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229092 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229096 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229099 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229102 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229104 2555 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229107 2555 flags.go:64] FLAG: --enable-server="true" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229110 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229114 2555 flags.go:64] FLAG: --event-burst="100" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229118 2555 flags.go:64] FLAG: --event-qps="50" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229121 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229123 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229126 2555 flags.go:64] FLAG: --eviction-hard="" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229134 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229137 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229140 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229143 2555 flags.go:64] FLAG: --eviction-soft="" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229145 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229148 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 04:33:05.236487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229151 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229154 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229157 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229160 2555 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229163 2555 flags.go:64] FLAG: --feature-gates="" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229166 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229169 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229172 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229175 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229178 2555 flags.go:64] FLAG: --healthz-port="10248" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229180 2555 flags.go:64] FLAG: --help="false" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229183 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229186 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229190 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229193 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229196 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229199 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229202 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229205 2555 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229207 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229210 2555 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229213 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229216 2555 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229218 2555 flags.go:64] FLAG: --kube-reserved="" Jul 22 04:33:05.237344 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229221 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229224 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229227 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229229 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229232 2555 flags.go:64] FLAG: --lock-file="" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229234 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229237 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229240 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229245 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229247 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229251 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229253 2555 flags.go:64] FLAG: --logging-format="text" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229256 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229259 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229262 2555 flags.go:64] FLAG: --manifest-url="" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229264 2555 flags.go:64] FLAG: --manifest-url-header="" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229268 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229271 2555 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229275 2555 flags.go:64] FLAG: --max-pods="110" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229277 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229280 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229284 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229288 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229291 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229293 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 04:33:05.238180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229296 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229303 2555 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229305 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229308 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229311 2555 flags.go:64] FLAG: --pod-cidr="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229314 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229320 2555 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229322 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229325 2555 flags.go:64] FLAG: --pods-per-core="0" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229328 2555 flags.go:64] FLAG: --port="10250" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229331 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229333 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ef81edd9000bcb16" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229336 2555 flags.go:64] FLAG: --qos-reserved="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229339 2555 flags.go:64] FLAG: --read-only-port="10255" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229342 2555 flags.go:64] FLAG: --register-node="true" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229344 2555 flags.go:64] FLAG: --register-with-taints="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229348 2555 flags.go:64] FLAG: --registry-burst="10" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229350 2555 flags.go:64] FLAG: --registry-qps="5" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229353 2555 flags.go:64] FLAG: --reserved-cpus="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229356 2555 flags.go:64] FLAG: --reserved-memory="" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229359 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229362 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229365 2555 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229368 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229370 2555 flags.go:64] FLAG: --runonce="false" Jul 22 04:33:05.239060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229373 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229376 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229380 2555 flags.go:64] FLAG: --seccomp-default="false" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229384 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229387 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229390 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229393 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229396 2555 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229398 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229401 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229404 2555 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229406 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229410 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229412 2555 flags.go:64] FLAG: --system-cgroups="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229415 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229420 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229423 2555 flags.go:64] FLAG: --tls-cert-file="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229426 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229429 2555 flags.go:64] FLAG: --tls-min-version="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229432 2555 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229435 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229437 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229440 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229443 2555 flags.go:64] FLAG: --v="2" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229447 2555 flags.go:64] FLAG: --version="false" Jul 22 04:33:05.239900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229450 2555 flags.go:64] FLAG: --vmodule="" Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229454 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.229457 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229559 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229563 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229566 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229568 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229571 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229573 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229577 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229581 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229583 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229586 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229588 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229591 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229593 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229596 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229598 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229600 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229603 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.240771 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229606 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229609 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229611 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229613 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229616 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229618 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229621 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229623 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229626 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229628 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229631 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229633 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229636 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229638 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229641 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229644 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229647 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229650 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229654 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229656 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.241505 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229659 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229663 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229667 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229669 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229673 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229677 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229680 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229683 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229685 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229688 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229690 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229693 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229695 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229698 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229701 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229703 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229706 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229708 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229711 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229713 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.242296 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229715 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229718 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229721 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229723 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229726 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229730 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229733 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229736 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229739 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229742 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229745 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229748 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229750 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229754 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229761 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229763 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229766 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229768 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229771 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.242975 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229774 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229776 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229779 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229781 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229784 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229796 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229798 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229801 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229803 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229806 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229809 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229811 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229814 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229817 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229819 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229822 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229824 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229827 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229829 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229832 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.243620 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229834 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229838 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.229840 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.230369 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.236338 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.236351 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.236367 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.236376 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236426 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236430 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236434 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236437 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236440 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236442 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.244303 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236445 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236448 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236450 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236453 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236455 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236458 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236461 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236463 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236466 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236468 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236471 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236473 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236476 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236478 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236481 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236483 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236485 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236488 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236490 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236493 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.244793 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236496 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236499 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236501 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236505 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236507 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236510 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236512 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236515 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236517 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236520 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236522 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236525 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236527 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236529 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236532 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236534 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236537 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236539 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236542 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236544 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.245466 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236546 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236549 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236551 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236554 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236556 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236558 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236561 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236563 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236566 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236568 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236571 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236573 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236576 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236579 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236582 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236585 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236588 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236590 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236593 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236595 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.246282 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236597 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236600 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236603 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236607 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236610 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236613 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236616 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236619 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236621 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236624 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236626 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236629 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236632 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236635 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236638 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236640 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236642 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236645 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236648 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.246821 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236652 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236656 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236659 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236662 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236665 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236667 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236673 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236675 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236679 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236682 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236684 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236687 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236689 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236692 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.247308 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.236697 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236795 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236800 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236802 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236805 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236808 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236811 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236814 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236816 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236819 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236822 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236824 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236827 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236830 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236832 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236835 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236839 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236842 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236844 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:33:05.247653 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236847 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236850 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236852 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236855 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236858 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236860 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236863 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236866 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236869 2555 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236871 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236874 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236876 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236879 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236881 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236883 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236885 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236888 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236890 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236892 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236895 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:33:05.248210 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236897 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236900 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236902 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236904 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236907 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236909 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236911 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236914 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236916 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236918 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236921 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236923 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236925 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236928 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236930 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236933 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236936 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236939 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236942 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:33:05.248721 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236945 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236948 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236950 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236953 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236955 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236958 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236960 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236962 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236965 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236967 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236970 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236972 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236975 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236977 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236980 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236982 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236984 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236987 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236989 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236991 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:33:05.249203 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236994 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236996 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.236998 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237000 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237003 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237006 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237008 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237010 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237013 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237015 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237018 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237038 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237041 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237045 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237047 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237050 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237052 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237055 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237057 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237060 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:33:05.249725 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237062 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:05.237064 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.237069 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.237149 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.239268 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.240845 2555 server.go:1019] "Starting client certificate rotation" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.240940 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.240968 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.243209 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 04:33:05.250223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.244200 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 04:33:05.256568 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.256549 2555 log.go:25] "Validated CRI v1 runtime API" Jul 22 04:33:05.259567 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.259552 2555 log.go:25] "Validated CRI v1 image API" Jul 22 04:33:05.260375 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.260357 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 04:33:05.263164 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.263144 2555 fs.go:135] Filesystem UUIDs: map[636cb89e-4704-463b-95dd-4568a8237ee1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ec8fb271-fbe4-48cf-a54a-a9e93231b10b:/dev/nvme0n1p3] Jul 22 04:33:05.263235 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.263162 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 04:33:05.267733 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.267639 2555 manager.go:216] Machine: {Timestamp:2026-07-22 04:33:05.267197885 +0000 UTC m=+0.315466134 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200211 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27e133deb9ca4470bfed466dc8af9d SystemUUID:ec27e133-deb9-ca44-70bf-ed466dc8af9d BootID:5b968bcc-34bc-4c98-92c8-14f93e6cca60 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:f9:c8:21:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:f9:c8:21:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:76:2b:17:c7:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 04:33:05.267733 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.267722 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 04:33:05.267853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.267778 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 04:33:05.268753 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.268733 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 04:33:05.268887 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.268756 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-75.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 04:33:05.270138 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.270127 2555 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 04:33:05.270171 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.270141 2555 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 04:33:05.270171 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.270148 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 04:33:05.270715 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.270699 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 04:33:05.270894 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.270884 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 04:33:05.272603 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.272592 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 04:33:05.275857 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.275847 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 22 04:33:05.275977 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.275968 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 04:33:05.276006 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.275996 2555 kubelet.go:486] "Attempting to sync node with API server" Jul 22 04:33:05.276006 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276006 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 04:33:05.276080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276018 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 04:33:05.276080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276039 2555 kubelet.go:398] "Adding apiserver pod source" Jul 22 04:33:05.276080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276050 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 04:33:05.276358 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276345 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 04:33:05.276404 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.276366 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 04:33:05.278528 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.278504 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 04:33:05.280803 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.280790 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 04:33:05.280855 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.280819 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 04:33:05.281686 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281675 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 04:33:05.281723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281691 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 04:33:05.281723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281697 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 04:33:05.281723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281702 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 04:33:05.281723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281709 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 04:33:05.281723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281714 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281745 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281751 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281758 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281763 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281770 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 04:33:05.281868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.281778 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 04:33:05.282435 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.282425 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 04:33:05.282435 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.282435 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 04:33:05.285143 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.285126 2555 server.go:1268] "Started kubelet" Jul 22 04:33:05.285249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.285200 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 04:33:05.285293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.285275 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 04:33:05.285332 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.285267 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 04:33:05.285802 ip-10-0-133-75 systemd[1]: Started Kubernetes Kubelet. Jul 22 04:33:05.286606 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.286551 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 04:33:05.286606 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.286554 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 04:33:05.286606 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.286584 2555 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-75.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 04:33:05.287173 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.287156 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 04:33:05.288003 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.287988 2555 server.go:310] "Adding debug handlers to kubelet server" Jul 22 04:33:05.291416 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.291376 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 04:33:05.292411 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.291430 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-75.ec2.internal.18c48273956918fa default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-75.ec2.internal,UID:ip-10-0-133-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-75.ec2.internal,},FirstTimestamp:2026-07-22 04:33:05.285101818 +0000 UTC m=+0.333370067,LastTimestamp:2026-07-22 04:33:05.285101818 +0000 UTC m=+0.333370067,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-75.ec2.internal,}" Jul 22 04:33:05.297226 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297208 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 04:33:05.297226 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297219 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 04:33:05.297328 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.297300 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.297891 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297873 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 04:33:05.297970 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297919 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 04:33:05.297970 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297930 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 04:33:05.298091 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.297987 2555 factory.go:55] Registering systemd factory Jul 22 04:33:05.298091 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298001 2555 factory.go:223] Registration of the systemd container factory successfully Jul 22 04:33:05.298225 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298200 2555 factory.go:153] Registering CRI-O factory Jul 22 04:33:05.298316 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298228 2555 factory.go:223] Registration of the crio container factory successfully Jul 22 04:33:05.298316 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298284 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 04:33:05.298316 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298306 2555 factory.go:103] Registering Raw factory Jul 22 04:33:05.298459 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298322 2555 manager.go:1196] Started watching for new ooms in manager Jul 22 04:33:05.298785 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.298725 2555 manager.go:349] Starting recovery of all containers Jul 22 04:33:05.299096 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.299048 2555 reconstruct.go:97] "Volume reconstruction finished" Jul 22 04:33:05.299096 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.299065 2555 reconciler.go:29] "Reconciler: start to sync state" Jul 22 04:33:05.299695 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.299671 2555 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 04:33:05.299784 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.299759 2555 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-75.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 04:33:05.308480 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.308463 2555 manager.go:354] Recovery completed Jul 22 04:33:05.309623 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.309607 2555 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 22 04:33:05.312440 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.312428 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.314342 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314320 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.314411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314351 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.314411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314360 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.314915 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314901 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 04:33:05.314915 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314910 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.315005 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314913 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 04:33:05.315005 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.314957 2555 state_mem.go:36] "Initialized new in-memory state store" Jul 22 04:33:05.315592 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.315578 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.315644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.315604 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.315644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.315619 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.316463 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.316451 2555 policy_none.go:49] "None policy: Start" Jul 22 04:33:05.316510 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.316469 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 04:33:05.316510 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.316479 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 04:33:05.317600 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.317588 2555 policy_none.go:47] "Start" Jul 22 04:33:05.317716 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.317701 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8gdjv" Jul 22 04:33:05.323399 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.323271 2555 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-75.ec2.internal.18c4827397273d01 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-75.ec2.internal,UID:ip-10-0-133-75.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-75.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-75.ec2.internal,},FirstTimestamp:2026-07-22 04:33:05.314340097 +0000 UTC m=+0.362608346,LastTimestamp:2026-07-22 04:33:05.314340097 +0000 UTC m=+0.362608346,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-75.ec2.internal,}" Jul 22 04:33:05.324949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.324935 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8gdjv" Jul 22 04:33:05.354548 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354531 2555 manager.go:337] "Starting Device Plugin manager" Jul 22 04:33:05.354622 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.354579 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 04:33:05.354622 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354591 2555 server.go:87] "Starting device plugin registration server" Jul 22 04:33:05.354807 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354795 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 04:33:05.354864 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354807 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 04:33:05.354914 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354889 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 04:33:05.354975 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354963 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 04:33:05.355043 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.354975 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 04:33:05.355434 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.355412 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 04:33:05.355530 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.355457 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.422941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.422923 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 04:33:05.423986 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.423971 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 04:33:05.424083 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.424000 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 04:33:05.424083 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.424015 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 04:33:05.424083 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.424058 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 04:33:05.426094 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.426073 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:05.455707 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.455689 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.456903 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.456888 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.456966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.456912 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.456966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.456922 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.456966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.456940 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.466540 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.466521 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.466540 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.466539 2555 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-75.ec2.internal\": node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.489242 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.489201 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.524747 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.524728 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal"] Jul 22 04:33:05.524794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.524787 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.525489 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.525477 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.525549 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.525501 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.525549 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.525516 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.526623 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.526610 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.526747 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.526732 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.526781 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.526758 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.527298 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527275 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.527298 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527292 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.527416 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527305 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.527416 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527311 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.527416 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527319 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.527416 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.527321 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.528399 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.528383 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.528475 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.528412 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:05.529050 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.529015 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:05.529130 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.529060 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:05.529130 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.529075 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:05.542568 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.542553 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-75.ec2.internal\" not found" node="ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.546077 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.546060 2555 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-75.ec2.internal\" not found" node="ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.589458 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.589437 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.599686 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.599670 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.599760 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.599691 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.599760 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.599707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8a511dbb64386472f7853b93500b3424-config\") pod \"kube-apiserver-proxy-ip-10-0-133-75.ec2.internal\" (UID: \"8a511dbb64386472f7853b93500b3424\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.689686 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.689653 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.700001 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.699975 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.700104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.700010 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8a511dbb64386472f7853b93500b3424-config\") pod \"kube-apiserver-proxy-ip-10-0-133-75.ec2.internal\" (UID: \"8a511dbb64386472f7853b93500b3424\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.700104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.700052 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.700104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.700088 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8a511dbb64386472f7853b93500b3424-config\") pod \"kube-apiserver-proxy-ip-10-0-133-75.ec2.internal\" (UID: \"8a511dbb64386472f7853b93500b3424\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.700104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.700102 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.700243 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.700105 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/610d9e0ff95d95bca94ab45c88df2393-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal\" (UID: \"610d9e0ff95d95bca94ab45c88df2393\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.790811 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.790787 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.847588 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.847563 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.851800 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:05.851731 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:05.891343 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.891315 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:05.998299 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:05.998262 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:06.104720 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:06.104660 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:06.117842 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.117819 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:06.204755 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:06.204725 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:06.240452 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.240425 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 04:33:06.240824 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.240547 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 04:33:06.240824 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.240603 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 04:33:06.305768 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:06.305747 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:06.329593 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.329556 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 04:28:05 +0000 UTC" deadline="2028-03-26 14:52:33.469337431 +0000 UTC" Jul 22 04:33:06.329685 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.329599 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14722h19m27.139753258s" Jul 22 04:33:06.379153 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.379128 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:33:06.381854 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:06.381813 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a511dbb64386472f7853b93500b3424.slice/crio-8f35cc27d15298f2aaee39bad34b4b854e9f1b2dcf7d964572e6f461ec5bf6d1 WatchSource:0}: Error finding container 8f35cc27d15298f2aaee39bad34b4b854e9f1b2dcf7d964572e6f461ec5bf6d1: Status 404 returned error can't find the container with id 8f35cc27d15298f2aaee39bad34b4b854e9f1b2dcf7d964572e6f461ec5bf6d1 Jul 22 04:33:06.382375 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.382359 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:33:06.382459 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.382394 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:33:06.382459 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.382409 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:33:06.386927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.386911 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:33:06.406821 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:06.406799 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-75.ec2.internal\" not found" Jul 22 04:33:06.427267 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.427229 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" event={"ID":"8a511dbb64386472f7853b93500b3424","Type":"ContainerStarted","Data":"8f35cc27d15298f2aaee39bad34b4b854e9f1b2dcf7d964572e6f461ec5bf6d1"} Jul 22 04:33:06.438349 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:06.438326 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod610d9e0ff95d95bca94ab45c88df2393.slice/crio-ad94462985da20a726354fc8d220a207f9d747a5ee4cf9558e3aad34e9103184 WatchSource:0}: Error finding container ad94462985da20a726354fc8d220a207f9d747a5ee4cf9558e3aad34e9103184: Status 404 returned error can't find the container with id ad94462985da20a726354fc8d220a207f9d747a5ee4cf9558e3aad34e9103184 Jul 22 04:33:06.473930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.473915 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:06.498276 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.498253 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" Jul 22 04:33:06.511526 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.511505 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 04:33:06.513306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.513293 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" Jul 22 04:33:06.522124 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.522109 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 04:33:06.761012 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:06.760951 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:07.277483 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.277461 2555 apiserver.go:52] "Watching apiserver" Jul 22 04:33:07.287127 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.287102 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 04:33:07.289992 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.289148 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj","openshift-cluster-node-tuning-operator/tuned-4vkwr","openshift-image-registry/node-ca-wfvlw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal","openshift-multus/multus-69mng","openshift-multus/multus-additional-cni-plugins-nzzlb","openshift-multus/network-metrics-daemon-2bwfx","kube-system/global-pull-secret-syncer-95d2d","kube-system/konnectivity-agent-wpln7","openshift-dns/node-resolver-7266c","openshift-network-diagnostics/network-check-target-m2gjh","openshift-network-operator/iptables-alerter-kfddv","openshift-ovn-kubernetes/ovnkube-node-5752d"] Jul 22 04:33:07.291180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.291154 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:07.291296 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.291271 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:07.292779 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.292757 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.293754 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.293731 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.294924 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.294895 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.295637 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.295609 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.295738 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.295609 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.295738 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.295711 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 04:33:07.295835 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.295797 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dj76d\"" Jul 22 04:33:07.296054 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.296021 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.296360 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.296244 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.296664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.296640 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.296755 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.296730 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-czspn\"" Jul 22 04:33:07.297364 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297338 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69mng" Jul 22 04:33:07.297545 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297425 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 04:33:07.297545 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297439 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 04:33:07.297545 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297462 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.297718 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297553 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.297718 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.297681 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w92rg\"" Jul 22 04:33:07.298432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.298407 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.298599 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.298543 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 04:33:07.298695 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.298613 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.298695 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.298681 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 04:33:07.298801 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.298728 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:07.299023 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.299002 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-75jxj\"" Jul 22 04:33:07.299112 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.299050 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 04:33:07.299542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.299523 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.299933 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.299906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.300027 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.299991 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:07.300116 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.300101 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4kz8c\"" Jul 22 04:33:07.300179 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.300101 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 04:33:07.301121 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.301092 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.302366 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.302342 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.303508 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.303485 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.303611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.303541 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f99sg\"" Jul 22 04:33:07.303611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.303572 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 04:33:07.303717 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.303625 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 04:33:07.304709 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.304687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.304963 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.304935 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.304963 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.304961 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.306252 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.306233 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q4mvr\"" Jul 22 04:33:07.306775 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.306751 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.306897 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.306879 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cbz8r\"" Jul 22 04:33:07.306961 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.306902 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 04:33:07.306999 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.306965 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.307132 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.307109 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 04:33:07.307283 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.307266 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 04:33:07.308193 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.308122 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p287c\"" Jul 22 04:33:07.308297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.308275 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 04:33:07.309163 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.309140 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 04:33:07.309268 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.309188 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 04:33:07.309268 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.309256 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 04:33:07.311464 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.311445 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 04:33:07.313085 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313063 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-multus\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313099 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-hostroot\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313164 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-daemon-config\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313190 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.313310 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313213 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysconfig\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313310 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313236 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-lib-modules\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313310 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313270 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-host\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313421 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313313 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.313421 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313348 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-socket-dir-parent\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313421 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313397 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc285cbd-f28d-4803-bcf3-9346956fc406-iptables-alerter-script\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.313547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313421 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-run\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313461 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-var-lib-kubelet\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313491 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-os-release\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.313547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313513 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.313547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313532 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc285cbd-f28d-4803-bcf3-9346956fc406-host-slash\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313553 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313611 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-conf\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-sys\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-k8s-cni-cncf-io\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313708 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-netns\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313730 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-bin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.313761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313754 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-kubelet\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313794 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-system-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313820 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-cnibin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-os-release\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-multus-certs\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313935 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-systemd\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313954 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-tmp\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313977 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64d5801c-fe21-44dd-af4e-de832e04007c-serviceca\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.313999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cnibin\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.314015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314014 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314028 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314085 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmn6w\" (UniqueName: \"kubernetes.io/projected/5dd21a8c-358a-4a4a-adff-48c634977ea3-kube-api-access-zmn6w\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314142 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4thg\" (UniqueName: \"kubernetes.io/projected/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-kube-api-access-s4thg\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkpb9\" (UniqueName: \"kubernetes.io/projected/cc285cbd-f28d-4803-bcf3-9346956fc406-kube-api-access-xkpb9\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314197 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-kubernetes\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-tuned\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314310 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qxj\" (UniqueName: \"kubernetes.io/projected/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-kube-api-access-j7qxj\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-cni-binary-copy\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314363 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-conf-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314386 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-etc-kubernetes\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314409 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krtkr\" (UniqueName: \"kubernetes.io/projected/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-kube-api-access-krtkr\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314451 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d5801c-fe21-44dd-af4e-de832e04007c-host\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314483 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pzjp\" (UniqueName: \"kubernetes.io/projected/64d5801c-fe21-44dd-af4e-de832e04007c-kube-api-access-5pzjp\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.314512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314513 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-system-cni-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.315042 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314539 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-modprobe-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.315042 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.314573 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:07.332877 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.332857 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mvqj9" Jul 22 04:33:07.344026 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.344001 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mvqj9" Jul 22 04:33:07.398438 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.398422 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 04:33:07.414804 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414784 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-bin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.414906 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414816 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-kubelet\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.414906 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414878 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-os-release\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.414906 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414896 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-multus-certs\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414904 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-bin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414929 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-tmp\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414953 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cnibin\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414980 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd9108c-5501-48c3-a5ea-879f5e0a727f-tmp-dir\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414984 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-kubelet\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.414995 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-os-release\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415003 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-systemd-units\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415053 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-systemd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.415075 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415072 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cnibin\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415091 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmn6w\" (UniqueName: \"kubernetes.io/projected/5dd21a8c-358a-4a4a-adff-48c634977ea3-kube-api-access-zmn6w\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415112 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-multus-certs\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415119 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4thg\" (UniqueName: \"kubernetes.io/projected/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-kube-api-access-s4thg\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-kubernetes\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415223 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-kubernetes\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415328 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-tuned\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415359 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415364 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415428 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qxj\" (UniqueName: \"kubernetes.io/projected/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-kube-api-access-j7qxj\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-cni-binary-copy\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415501 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-conf-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415530 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-etc-kubernetes\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415578 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krtkr\" (UniqueName: \"kubernetes.io/projected/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-kube-api-access-krtkr\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.415597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415603 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d5801c-fe21-44dd-af4e-de832e04007c-host\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415629 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pzjp\" (UniqueName: \"kubernetes.io/projected/64d5801c-fe21-44dd-af4e-de832e04007c-kube-api-access-5pzjp\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415666 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-conf-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415675 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-registration-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415666 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-etc-kubernetes\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-modprobe-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415744 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64d5801c-fe21-44dd-af4e-de832e04007c-host\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415747 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415776 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31aaf12f-2127-4cb9-8d65-959776536a9e-agent-certs\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415798 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31aaf12f-2127-4cb9-8d65-959776536a9e-konnectivity-ca\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415822 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415829 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-modprobe-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415847 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-multus\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415872 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-lib-modules\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-host\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.416306 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-kubelet-config\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.415989 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416011 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-socket-dir-parent\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc285cbd-f28d-4803-bcf3-9346956fc406-iptables-alerter-script\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416125 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-lib-modules\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416152 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-cni-binary-copy\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416206 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-socket-dir-parent\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.416243 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416294 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-host\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.416339 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:07.916316761 +0000 UTC m=+2.964585014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416383 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-slash\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416413 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-config\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416460 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96l9j\" (UniqueName: \"kubernetes.io/projected/208deeb6-a463-4713-be0f-7699642be825-kube-api-access-96l9j\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416493 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc285cbd-f28d-4803-bcf3-9346956fc406-host-slash\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416494 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-cni-binary-copy\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416345 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-var-lib-cni-multus\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416538 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-conf\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417120 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416567 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc285cbd-f28d-4803-bcf3-9346956fc406-host-slash\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416652 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-sys\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416704 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-conf\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-var-lib-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-sys\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416752 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-log-socket\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416778 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-sys-fs\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416806 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-k8s-cni-cncf-io\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416833 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-system-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-cnibin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-systemd\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416908 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64d5801c-fe21-44dd-af4e-de832e04007c-serviceca\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416927 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-system-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416959 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-netd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416985 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-cnibin\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416882 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-k8s-cni-cncf-io\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.417990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.416984 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-script-lib\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417134 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417147 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc285cbd-f28d-4803-bcf3-9346956fc406-iptables-alerter-script\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417161 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-dbus\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417186 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417215 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27f6b\" (UniqueName: \"kubernetes.io/projected/4cd9108c-5501-48c3-a5ea-879f5e0a727f-kube-api-access-27f6b\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417238 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-systemd\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417243 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-device-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417284 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkpb9\" (UniqueName: \"kubernetes.io/projected/cc285cbd-f28d-4803-bcf3-9346956fc406-kube-api-access-xkpb9\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417349 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-node-log\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417373 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-env-overrides\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd9108c-5501-48c3-a5ea-879f5e0a727f-hosts-file\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417412 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417419 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-system-cni-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.418751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417460 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-netns\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-etc-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417496 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-cni-dir\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417496 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/64d5801c-fe21-44dd-af4e-de832e04007c-serviceca\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417514 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-bin\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417544 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417566 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-system-cni-dir\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417591 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-ovn\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417613 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417632 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-hostroot\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-daemon-config\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417681 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysconfig\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417715 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-run\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417734 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-var-lib-kubelet\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417739 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-hostroot\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417754 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-os-release\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417785 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.419625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysconfig\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417814 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52634ac6-7c8f-4615-8a35-33994da99ed7-ovn-node-metrics-cert\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d9m4\" (UniqueName: \"kubernetes.io/projected/52634ac6-7c8f-4615-8a35-33994da99ed7-kube-api-access-5d9m4\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417821 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-run\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417868 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-os-release\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417921 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-var-lib-kubelet\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417969 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-sysctl-d\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.417976 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-kubelet\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.418006 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-socket-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.418056 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-netns\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.418164 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5dd21a8c-358a-4a4a-adff-48c634977ea3-host-run-netns\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.418347 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5dd21a8c-358a-4a4a-adff-48c634977ea3-multus-daemon-config\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.418459 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.419518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-etc-tuned\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.420495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.419564 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-tmp\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.422968 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.422946 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:07.423064 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.422973 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:07.423064 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.422987 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:07.423195 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.423099 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:07.923081036 +0000 UTC m=+2.971349296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:07.424838 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.424623 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4thg\" (UniqueName: \"kubernetes.io/projected/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-kube-api-access-s4thg\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.424838 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.424788 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmn6w\" (UniqueName: \"kubernetes.io/projected/5dd21a8c-358a-4a4a-adff-48c634977ea3-kube-api-access-zmn6w\") pod \"multus-69mng\" (UID: \"5dd21a8c-358a-4a4a-adff-48c634977ea3\") " pod="openshift-multus/multus-69mng" Jul 22 04:33:07.426655 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.426628 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkpb9\" (UniqueName: \"kubernetes.io/projected/cc285cbd-f28d-4803-bcf3-9346956fc406-kube-api-access-xkpb9\") pod \"iptables-alerter-kfddv\" (UID: \"cc285cbd-f28d-4803-bcf3-9346956fc406\") " pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.426950 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.426926 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krtkr\" (UniqueName: \"kubernetes.io/projected/e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd-kube-api-access-krtkr\") pod \"tuned-4vkwr\" (UID: \"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd\") " pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.427049 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.427014 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qxj\" (UniqueName: \"kubernetes.io/projected/b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4-kube-api-access-j7qxj\") pod \"multus-additional-cni-plugins-nzzlb\" (UID: \"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4\") " pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.427392 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.427370 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pzjp\" (UniqueName: \"kubernetes.io/projected/64d5801c-fe21-44dd-af4e-de832e04007c-kube-api-access-5pzjp\") pod \"node-ca-wfvlw\" (UID: \"64d5801c-fe21-44dd-af4e-de832e04007c\") " pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.430029 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.429983 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" event={"ID":"610d9e0ff95d95bca94ab45c88df2393","Type":"ContainerStarted","Data":"ad94462985da20a726354fc8d220a207f9d747a5ee4cf9558e3aad34e9103184"} Jul 22 04:33:07.519130 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519101 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31aaf12f-2127-4cb9-8d65-959776536a9e-agent-certs\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.519233 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31aaf12f-2127-4cb9-8d65-959776536a9e-konnectivity-ca\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.519289 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519258 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519334 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-kubelet-config\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.519334 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-slash\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519439 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519353 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-config\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519439 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519378 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96l9j\" (UniqueName: \"kubernetes.io/projected/208deeb6-a463-4713-be0f-7699642be825-kube-api-access-96l9j\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519439 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-slash\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519439 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519405 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519473 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-kubelet-config\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519405 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-var-lib-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519516 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-log-socket\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519545 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-sys-fs\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519584 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519656 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-netd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519633 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-script-lib\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519687 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-sys-fs\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519686 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-dbus\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519740 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-netd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519743 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519768 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27f6b\" (UniqueName: \"kubernetes.io/projected/4cd9108c-5501-48c3-a5ea-879f5e0a727f-kube-api-access-27f6b\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519763 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519793 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-device-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519831 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-device-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-node-log\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519853 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/31aaf12f-2127-4cb9-8d65-959776536a9e-konnectivity-ca\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519870 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-env-overrides\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd9108c-5501-48c3-a5ea-879f5e0a727f-hosts-file\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-netns\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519914 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-node-log\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.519949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-etc-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519931 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-config\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519946 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-etc-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-bin\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519978 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-var-lib-openvswitch\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519980 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519990 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4cd9108c-5501-48c3-a5ea-879f5e0a727f-hosts-file\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520012 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-netns\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-ovn\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520034 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-cni-bin\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.519720 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-log-socket\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520039 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520059 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52634ac6-7c8f-4615-8a35-33994da99ed7-ovn-node-metrics-cert\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d9m4\" (UniqueName: \"kubernetes.io/projected/52634ac6-7c8f-4615-8a35-33994da99ed7-kube-api-access-5d9m4\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520073 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520127 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-kubelet\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.520794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520131 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-ovn\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520161 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-socket-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520171 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-kubelet\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520193 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd9108c-5501-48c3-a5ea-879f5e0a727f-tmp-dir\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520212 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-host-run-ovn-kubernetes\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-systemd-units\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-systemd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520313 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-socket-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-registration-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520353 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-ovnkube-script-lib\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520353 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-systemd-units\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520401 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/208deeb6-a463-4713-be0f-7699642be825-registration-dir\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520453 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52634ac6-7c8f-4615-8a35-33994da99ed7-env-overrides\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.520465 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520446 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-dbus\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520504 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52634ac6-7c8f-4615-8a35-33994da99ed7-run-systemd\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.520515 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:08.020499816 +0000 UTC m=+3.068768071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:07.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.520624 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd9108c-5501-48c3-a5ea-879f5e0a727f-tmp-dir\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.522382 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.521806 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/31aaf12f-2127-4cb9-8d65-959776536a9e-agent-certs\") pod \"konnectivity-agent-wpln7\" (UID: \"31aaf12f-2127-4cb9-8d65-959776536a9e\") " pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.522640 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.522618 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52634ac6-7c8f-4615-8a35-33994da99ed7-ovn-node-metrics-cert\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.529248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.529199 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d9m4\" (UniqueName: \"kubernetes.io/projected/52634ac6-7c8f-4615-8a35-33994da99ed7-kube-api-access-5d9m4\") pod \"ovnkube-node-5752d\" (UID: \"52634ac6-7c8f-4615-8a35-33994da99ed7\") " pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.530019 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.529995 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27f6b\" (UniqueName: \"kubernetes.io/projected/4cd9108c-5501-48c3-a5ea-879f5e0a727f-kube-api-access-27f6b\") pod \"node-resolver-7266c\" (UID: \"4cd9108c-5501-48c3-a5ea-879f5e0a727f\") " pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.530360 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.530338 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96l9j\" (UniqueName: \"kubernetes.io/projected/208deeb6-a463-4713-be0f-7699642be825-kube-api-access-96l9j\") pod \"aws-ebs-csi-driver-node-4w7dj\" (UID: \"208deeb6-a463-4713-be0f-7699642be825\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.551566 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.551546 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:07.605403 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.605239 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kfddv" Jul 22 04:33:07.611787 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.611764 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" Jul 22 04:33:07.619553 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:07.619526 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc285cbd_f28d_4803_bcf3_9346956fc406.slice/crio-1e1ef0a389195e893187c364e8e1b84cb022f61001ec8b3108726992dc3db9ff WatchSource:0}: Error finding container 1e1ef0a389195e893187c364e8e1b84cb022f61001ec8b3108726992dc3db9ff: Status 404 returned error can't find the container with id 1e1ef0a389195e893187c364e8e1b84cb022f61001ec8b3108726992dc3db9ff Jul 22 04:33:07.620700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.620680 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wfvlw" Jul 22 04:33:07.624284 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:07.624262 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d9b2fc_c895_406f_bfc4_0b6a1bbb89bd.slice/crio-6a81d44ea21f65f62567be6de729611e3457b5a4e8500b0d32065aebc2795392 WatchSource:0}: Error finding container 6a81d44ea21f65f62567be6de729611e3457b5a4e8500b0d32065aebc2795392: Status 404 returned error can't find the container with id 6a81d44ea21f65f62567be6de729611e3457b5a4e8500b0d32065aebc2795392 Jul 22 04:33:07.627755 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.627736 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" Jul 22 04:33:07.629647 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:07.629623 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64d5801c_fe21_44dd_af4e_de832e04007c.slice/crio-bf7c19a0168366e405650517f88ed61d2207a8ed23d72ee3b13877b349a9ae23 WatchSource:0}: Error finding container bf7c19a0168366e405650517f88ed61d2207a8ed23d72ee3b13877b349a9ae23: Status 404 returned error can't find the container with id bf7c19a0168366e405650517f88ed61d2207a8ed23d72ee3b13877b349a9ae23 Jul 22 04:33:07.632698 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.632678 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-69mng" Jul 22 04:33:07.638207 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.638188 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:07.643967 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.643944 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7266c" Jul 22 04:33:07.649780 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.649768 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" Jul 22 04:33:07.654565 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.654542 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:07.667335 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:07.667312 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod208deeb6_a463_4713_be0f_7699642be825.slice/crio-07a1a7e2eb943ac9b75567dfe0a56eb8f41199279267ff41223f7a1f66ce120a WatchSource:0}: Error finding container 07a1a7e2eb943ac9b75567dfe0a56eb8f41199279267ff41223f7a1f66ce120a: Status 404 returned error can't find the container with id 07a1a7e2eb943ac9b75567dfe0a56eb8f41199279267ff41223f7a1f66ce120a Jul 22 04:33:07.667997 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:07.667911 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52634ac6_7c8f_4615_8a35_33994da99ed7.slice/crio-f7c3a70a3ecf3f2a3a0f39fdbe501bf515abf5d13997967744903e1f41e60af2 WatchSource:0}: Error finding container f7c3a70a3ecf3f2a3a0f39fdbe501bf515abf5d13997967744903e1f41e60af2: Status 404 returned error can't find the container with id f7c3a70a3ecf3f2a3a0f39fdbe501bf515abf5d13997967744903e1f41e60af2 Jul 22 04:33:07.922683 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:07.922599 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:07.922833 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.922737 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:07.922833 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:07.922808 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:08.92278874 +0000 UTC m=+3.971056981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:08.023709 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.023488 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:08.023709 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.023562 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:08.023709 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023684 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:08.024008 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023722 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:08.024008 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023739 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:08.024008 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023688 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:08.024008 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023838 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:09.023780752 +0000 UTC m=+4.072048990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:08.024008 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.023932 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:09.023910782 +0000 UTC m=+4.072179037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:08.346680 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.346556 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 04:28:07 +0000 UTC" deadline="2028-03-25 15:41:55.608266079 +0000 UTC" Jul 22 04:33:08.346680 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.346590 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14699h8m47.261680439s" Jul 22 04:33:08.425292 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.425259 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:08.425451 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.425391 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:08.433251 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.433217 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" event={"ID":"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd","Type":"ContainerStarted","Data":"6a81d44ea21f65f62567be6de729611e3457b5a4e8500b0d32065aebc2795392"} Jul 22 04:33:08.434704 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.434653 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kfddv" event={"ID":"cc285cbd-f28d-4803-bcf3-9346956fc406","Type":"ContainerStarted","Data":"1e1ef0a389195e893187c364e8e1b84cb022f61001ec8b3108726992dc3db9ff"} Jul 22 04:33:08.436070 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.436045 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" event={"ID":"208deeb6-a463-4713-be0f-7699642be825","Type":"ContainerStarted","Data":"07a1a7e2eb943ac9b75567dfe0a56eb8f41199279267ff41223f7a1f66ce120a"} Jul 22 04:33:08.437470 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.437449 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wfvlw" event={"ID":"64d5801c-fe21-44dd-af4e-de832e04007c","Type":"ContainerStarted","Data":"bf7c19a0168366e405650517f88ed61d2207a8ed23d72ee3b13877b349a9ae23"} Jul 22 04:33:08.439497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.439474 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"f7c3a70a3ecf3f2a3a0f39fdbe501bf515abf5d13997967744903e1f41e60af2"} Jul 22 04:33:08.440802 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.440773 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7266c" event={"ID":"4cd9108c-5501-48c3-a5ea-879f5e0a727f","Type":"ContainerStarted","Data":"bf0a92220addc41f5811569e7654dee3c1f25ee0e653d6eeffcf4a1abc01bd48"} Jul 22 04:33:08.442049 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.441989 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wpln7" event={"ID":"31aaf12f-2127-4cb9-8d65-959776536a9e","Type":"ContainerStarted","Data":"9997da13337b0fd818a571e0fe74abc2c9eada15fc7a3cba2fd42a0e4b6cd2de"} Jul 22 04:33:08.443460 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.443435 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69mng" event={"ID":"5dd21a8c-358a-4a4a-adff-48c634977ea3","Type":"ContainerStarted","Data":"6bc88e2a560becf6e68cdcf3516098c9b31787168c280536fd1c87c6d5f3e8bf"} Jul 22 04:33:08.444824 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.444766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerStarted","Data":"7f82a0b53ce301f66c22182cdb553b086cc78ab97a82298b6be60663cd77e999"} Jul 22 04:33:08.473773 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.473747 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:33:08.930668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:08.930637 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:08.930875 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.930807 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:08.930938 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:08.930878 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:10.930859039 +0000 UTC m=+5.979127281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:09.031418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.031383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:09.031584 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.031457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:09.031674 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031596 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:09.031674 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031666 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:11.031636456 +0000 UTC m=+6.079904710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:09.031796 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031770 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:09.031850 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031806 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:09.031850 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031821 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:09.031962 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.031877 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:11.031859579 +0000 UTC m=+6.080127820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:09.347519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.347398 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 04:28:07 +0000 UTC" deadline="2028-02-03 00:25:46.744784642 +0000 UTC" Jul 22 04:33:09.347519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.347436 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13459h52m37.397353388s" Jul 22 04:33:09.429079 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.427458 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:09.429079 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.427625 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:09.434144 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:09.433097 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:09.434144 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:09.433394 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:10.424955 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:10.424927 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:10.425436 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:10.425071 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:10.948602 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:10.948572 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:10.948770 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:10.948729 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:10.948825 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:10.948792 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:14.948774591 +0000 UTC m=+9.997042827 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:11.049342 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:11.049310 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:11.049499 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:11.049381 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:11.049561 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.049501 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:11.049561 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.049553 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:15.049536527 +0000 UTC m=+10.097804775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:11.049961 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.049942 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:11.050102 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.049968 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:11.050102 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.049981 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:11.050102 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.050038 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:15.050009015 +0000 UTC m=+10.098277265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:11.427594 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:11.426877 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:11.427594 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.426997 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:11.427594 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:11.427393 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:11.427594 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:11.427492 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:12.425047 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:12.424531 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:12.425047 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:12.424669 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:13.425423 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:13.425202 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:13.425861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:13.425235 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:13.425861 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:13.425570 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:13.425974 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:13.425886 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:14.424936 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:14.424906 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:14.425125 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:14.425048 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:14.978924 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:14.978885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:14.979366 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:14.979041 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:14.979366 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:14.979105 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:22.979086925 +0000 UTC m=+18.027355180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:15.079496 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:15.079577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079715 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079770 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079788 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079802 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079772 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:23.079755056 +0000 UTC m=+18.128023306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:15.079900 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.079848 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:23.079836064 +0000 UTC m=+18.128104313 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:15.425632 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:15.425560 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:15.425779 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.425662 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:15.426203 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:15.426051 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:15.426203 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:15.426162 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:16.425128 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:16.425068 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:16.425464 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:16.425175 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:16.464625 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:16.464593 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" event={"ID":"610d9e0ff95d95bca94ab45c88df2393","Type":"ContainerStarted","Data":"69b79e2171107c4d42d46fff0036927fafbaa47f384ec7ea555c29fcd797d420"} Jul 22 04:33:17.424796 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:17.424767 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:17.424972 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:17.424896 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:17.425082 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:17.425059 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:17.425194 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:17.425176 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:17.467965 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:17.467928 2555 generic.go:356] "Generic (PLEG): container finished" podID="610d9e0ff95d95bca94ab45c88df2393" containerID="69b79e2171107c4d42d46fff0036927fafbaa47f384ec7ea555c29fcd797d420" exitCode=0 Jul 22 04:33:17.468138 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:17.467976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" event={"ID":"610d9e0ff95d95bca94ab45c88df2393","Type":"ContainerDied","Data":"69b79e2171107c4d42d46fff0036927fafbaa47f384ec7ea555c29fcd797d420"} Jul 22 04:33:18.425235 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:18.425204 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:18.425580 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:18.425325 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:19.425057 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:19.425008 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:19.425231 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:19.425209 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:19.425298 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:19.425264 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:19.425602 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:19.425379 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:20.424991 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:20.424956 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:20.425188 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:20.425099 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:21.425041 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:21.425002 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:21.425464 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:21.425007 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:21.425464 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:21.425172 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:21.425464 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:21.425222 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:22.424413 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:22.424330 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:22.424581 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:22.424428 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:23.044435 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:23.044395 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:23.044804 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.044548 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:23.044804 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.044607 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:33:39.044591826 +0000 UTC m=+34.092860062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:23.144681 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:23.144653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:23.144843 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:23.144713 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:23.144843 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144815 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:23.144843 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144819 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:23.144843 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144842 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:23.145037 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144852 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:23.145037 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144866 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:39.14485074 +0000 UTC m=+34.193118977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:23.145037 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.144887 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:39.144877559 +0000 UTC m=+34.193145799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:23.425226 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:23.425151 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:23.425226 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:23.425199 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:23.425414 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.425284 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:23.425466 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:23.425405 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:24.425077 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:24.425046 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:24.425438 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:24.425159 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:25.425230 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:25.425195 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:25.425682 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:25.425301 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:25.425682 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:25.425406 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:25.425682 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:25.425534 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:26.424718 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.424603 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:26.424801 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:26.424788 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:26.484365 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.484321 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" event={"ID":"8a511dbb64386472f7853b93500b3424","Type":"ContainerStarted","Data":"be7c53908ec2576c18a86564199426caf068f5df35c04ff9ef44fd2cfd154c85"} Jul 22 04:33:26.485808 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.485783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" event={"ID":"610d9e0ff95d95bca94ab45c88df2393","Type":"ContainerStarted","Data":"35c15d325d1e2f9a5eb0e1a86816af34e258c0d4d862b039857b2cdfe2d7055e"} Jul 22 04:33:26.486969 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.486947 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" event={"ID":"208deeb6-a463-4713-be0f-7699642be825","Type":"ContainerStarted","Data":"4103cfe3981df1628d3e636e9f2e5b6872e7dc09a09ee909a458809d81a79871"} Jul 22 04:33:26.488249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.488220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wfvlw" event={"ID":"64d5801c-fe21-44dd-af4e-de832e04007c","Type":"ContainerStarted","Data":"7f713cf8f7d3719aec104a0ae923e961305ceacc10e087f4d6eb52c7463396c4"} Jul 22 04:33:26.489522 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.489502 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7266c" event={"ID":"4cd9108c-5501-48c3-a5ea-879f5e0a727f","Type":"ContainerStarted","Data":"a2639ac7b2cea01b221ff3d2d4fcba3f94453e0be7f0b068729c4076e90f728f"} Jul 22 04:33:26.490854 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.490833 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wpln7" event={"ID":"31aaf12f-2127-4cb9-8d65-959776536a9e","Type":"ContainerStarted","Data":"5e74f29c9e91b8509a326eb38620dad9b4f5589d0f90b7d58dda3eee3036e97a"} Jul 22 04:33:26.492139 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.492115 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-69mng" event={"ID":"5dd21a8c-358a-4a4a-adff-48c634977ea3","Type":"ContainerStarted","Data":"a2b9b84212c8f1742cd82c08d54f1308eb463a56bcb4dc357bf716b9bda19c1a"} Jul 22 04:33:26.493389 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.493369 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerStarted","Data":"3bd5ed3db31692967eacb974a905a738bdf7e121ffb34a71c5feb67fb6fcbb9a"} Jul 22 04:33:26.494554 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.494534 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" event={"ID":"e7d9b2fc-c895-406f-bfc4-0b6a1bbb89bd","Type":"ContainerStarted","Data":"3ab99739150e497892d4d2be2ec00aa567ced0d2363dde060bb964fa662c8de1"} Jul 22 04:33:26.513517 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.513475 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-75.ec2.internal" podStartSLOduration=20.513463816 podStartE2EDuration="20.513463816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.49772213 +0000 UTC m=+21.545990387" watchObservedRunningTime="2026-07-22 04:33:26.513463816 +0000 UTC m=+21.561732073" Jul 22 04:33:26.513612 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.513594 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4vkwr" podStartSLOduration=2.94821699 podStartE2EDuration="21.513587959s" totalImagesPullingTime="18.565370969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.51305209 +0000 UTC m=+21.561320347" watchObservedRunningTime="2026-07-22 04:33:26.513587959 +0000 UTC m=+21.561856215" Jul 22 04:33:26.526063 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.526013 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wfvlw" podStartSLOduration=2.986319767 podStartE2EDuration="21.526001546s" totalImagesPullingTime="18.539681779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.525812148 +0000 UTC m=+21.574080429" watchObservedRunningTime="2026-07-22 04:33:26.526001546 +0000 UTC m=+21.574269803" Jul 22 04:33:26.563158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.563117 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-69mng" podStartSLOduration=2.005374901 podStartE2EDuration="20.563102032s" totalImagesPullingTime="18.557727131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.562744846 +0000 UTC m=+21.611013105" watchObservedRunningTime="2026-07-22 04:33:26.563102032 +0000 UTC m=+21.611370288" Jul 22 04:33:26.575966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.575915 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wpln7" podStartSLOduration=2.075337772 podStartE2EDuration="20.575900026s" totalImagesPullingTime="18.500562254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.57590013 +0000 UTC m=+21.624168380" watchObservedRunningTime="2026-07-22 04:33:26.575900026 +0000 UTC m=+21.624168277" Jul 22 04:33:26.590832 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.590795 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7266c" podStartSLOduration=2.098617619 podStartE2EDuration="20.590781312s" totalImagesPullingTime="18.492163693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.59066754 +0000 UTC m=+21.638935801" watchObservedRunningTime="2026-07-22 04:33:26.590781312 +0000 UTC m=+21.639049570" Jul 22 04:33:26.608243 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:26.608205 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-75.ec2.internal" podStartSLOduration=20.608191597 podStartE2EDuration="20.608191597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:26.607712119 +0000 UTC m=+21.655980376" watchObservedRunningTime="2026-07-22 04:33:26.608191597 +0000 UTC m=+21.656459855" Jul 22 04:33:27.424960 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.424771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:27.425085 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.424840 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:27.425148 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:27.425091 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:27.425204 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:27.425157 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:27.498224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.498198 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kfddv" event={"ID":"cc285cbd-f28d-4803-bcf3-9346956fc406","Type":"ContainerStarted","Data":"526fbfb7beb1b2fed996cd9e1b222995b884293f2d7afdc02fd0913b5d2e4963"} Jul 22 04:33:27.500579 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.500553 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" event={"ID":"208deeb6-a463-4713-be0f-7699642be825","Type":"ContainerStarted","Data":"ce3aa45c69d9444d42aa03a40a35cc8db199fe6bd6adedb1bd79a54bbc734192"} Jul 22 04:33:27.502153 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.502126 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 04:33:27.503544 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.503529 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:33:27.503880 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.503857 2555 generic.go:356] "Generic (PLEG): container finished" podID="52634ac6-7c8f-4615-8a35-33994da99ed7" containerID="28fa982e94afcb8af88879cffe5fcdd826d7d6432adbfc35509cf0a29b753862" exitCode=1 Jul 22 04:33:27.503948 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.503906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"f0c0c6eb25308f8d49d388e1f0ed9b4699373b0b508772f4b3229445b010729f"} Jul 22 04:33:27.504011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.503956 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"c8f3d06bb58cc2ba81a84a0465a5d3422fc2a21c7a2e18d3cd13fc9eaeada987"} Jul 22 04:33:27.504011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.503990 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"ae861028a2abc59f45f305da8b763de29b83c73067db0e7847d8ad6e17297215"} Jul 22 04:33:27.504125 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.504046 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"2d1678e06b65fc0a51e582635f02314d39813253ee7c2b13a5edb5cb72edcf54"} Jul 22 04:33:27.504125 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.504083 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerDied","Data":"28fa982e94afcb8af88879cffe5fcdd826d7d6432adbfc35509cf0a29b753862"} Jul 22 04:33:27.504125 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.504113 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"c92a0ab720342ee7320d21807dd538e7abbadef58dbe00c0644fe9a37937422f"} Jul 22 04:33:27.505357 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.505336 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="3bd5ed3db31692967eacb974a905a738bdf7e121ffb34a71c5feb67fb6fcbb9a" exitCode=0 Jul 22 04:33:27.505460 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.505425 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"3bd5ed3db31692967eacb974a905a738bdf7e121ffb34a71c5feb67fb6fcbb9a"} Jul 22 04:33:27.531318 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:27.531283 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kfddv" podStartSLOduration=3.99881739 podStartE2EDuration="22.531271262s" totalImagesPullingTime="18.532453872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:27.511588257 +0000 UTC m=+22.559856516" watchObservedRunningTime="2026-07-22 04:33:27.531271262 +0000 UTC m=+22.579539519" Jul 22 04:33:28.368362 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:28.368250 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T04:33:27.502150277Z","UUID":"e2b75e52-d880-4e68-8476-88b42e133556","Handler":null,"Name":"","Endpoint":""} Jul 22 04:33:28.370089 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:28.369987 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 04:33:28.370089 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:28.370045 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 04:33:28.424616 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:28.424589 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:28.424761 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:28.424735 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:29.424953 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:29.424782 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:29.425418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:29.424832 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:29.425418 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:29.425083 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:29.425418 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:29.425225 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:29.513745 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:29.513725 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:33:29.514175 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:29.514151 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"71f24d9246fe371ed484ec7bf20090436001d42ed9a2ad64c8f2c1004578d68a"} Jul 22 04:33:29.516763 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:29.516713 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" event={"ID":"208deeb6-a463-4713-be0f-7699642be825","Type":"ContainerStarted","Data":"53090d363d2fd27a7106c76489c88f031324177bf4cb613123752920af3976f3"} Jul 22 04:33:30.424189 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:30.424160 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:30.424364 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:30.424270 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:31.354642 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.354613 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:31.355286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.355245 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:31.370090 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.370035 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w7dj" podStartSLOduration=4.838681714 podStartE2EDuration="25.370002192s" totalImagesPullingTime="20.531320478s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:29.533653579 +0000 UTC m=+24.581921830" watchObservedRunningTime="2026-07-22 04:33:31.370002192 +0000 UTC m=+26.418270454" Jul 22 04:33:31.424337 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.424317 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:31.424470 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.424340 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:31.424470 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:31.424427 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:31.424585 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:31.424564 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:31.521682 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.521655 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:31.522288 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:31.522266 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wpln7" Jul 22 04:33:32.424441 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.424292 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:32.424862 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:32.424504 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:32.526327 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.526307 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:33:32.526715 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.526689 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"a5d988078d5878859b4317491ba0989914749b03ec609b51c614f4158fa32710"} Jul 22 04:33:32.527011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.526888 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:32.527011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.526961 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:32.527011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.527011 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:32.527237 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.527093 2555 scope.go:117] "RemoveContainer" containerID="28fa982e94afcb8af88879cffe5fcdd826d7d6432adbfc35509cf0a29b753862" Jul 22 04:33:32.528782 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.528759 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="1815faa4cb4a0b8f6b42769b22e1c8d5e2cd096d31d448eb0684fe1e251113de" exitCode=0 Jul 22 04:33:32.528865 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.528843 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"1815faa4cb4a0b8f6b42769b22e1c8d5e2cd096d31d448eb0684fe1e251113de"} Jul 22 04:33:32.543600 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.543572 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:32.543715 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:32.543703 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:33:33.424954 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.424749 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:33.425467 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.424749 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:33.425467 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:33.425087 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:33.425467 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:33.425196 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:33.534973 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.534954 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:33:33.535502 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.535378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" event={"ID":"52634ac6-7c8f-4615-8a35-33994da99ed7","Type":"ContainerStarted","Data":"8dab9e9c3424951b9b769da90c4be7854fc811f0574c4ebeceff05c9394f78e3"} Jul 22 04:33:33.537671 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.537647 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerStarted","Data":"e92aea7e46bcb8a3608c05d8da4ae350e6e3fbe27046deaa130f678b614aacde"} Jul 22 04:33:33.561525 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.561458 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" podStartSLOduration=8.64308976 podStartE2EDuration="27.561444738s" totalImagesPullingTime="18.918354978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:33.561053583 +0000 UTC m=+28.609321843" watchObservedRunningTime="2026-07-22 04:33:33.561444738 +0000 UTC m=+28.609712996" Jul 22 04:33:33.828832 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.828772 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-95d2d"] Jul 22 04:33:33.828938 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.828873 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:33.828977 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:33.828949 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:33.831705 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.831680 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bwfx"] Jul 22 04:33:33.831837 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.831792 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:33.831926 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:33.831896 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:33.832438 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.832416 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m2gjh"] Jul 22 04:33:33.832547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:33.832515 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:33.832627 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:33.832604 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:34.541442 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:34.541404 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="e92aea7e46bcb8a3608c05d8da4ae350e6e3fbe27046deaa130f678b614aacde" exitCode=0 Jul 22 04:33:34.541963 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:34.541485 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"e92aea7e46bcb8a3608c05d8da4ae350e6e3fbe27046deaa130f678b614aacde"} Jul 22 04:33:35.428398 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:35.428362 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:35.428562 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:35.428460 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:35.428562 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:35.428478 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:35.428562 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:35.428549 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:35.428731 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:35.428603 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:35.428731 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:35.428674 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:36.547564 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:36.547488 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="d69762e6a18441827dd84b0a4a1d5dcdfb3d90521e7c4360cedf22f818707c41" exitCode=0 Jul 22 04:33:36.547564 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:36.547536 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"d69762e6a18441827dd84b0a4a1d5dcdfb3d90521e7c4360cedf22f818707c41"} Jul 22 04:33:37.424962 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:37.424753 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:37.425137 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:37.424758 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:37.425137 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:37.425080 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-95d2d" podUID="a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8" Jul 22 04:33:37.425137 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:37.424807 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:37.425319 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:37.425152 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bwfx" podUID="379b2ddd-c7ae-4cda-ab35-996636b9eb3c" Jul 22 04:33:37.425319 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:37.425207 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m2gjh" podUID="284e065b-de22-4f10-b3a5-a17d61717871" Jul 22 04:33:39.067004 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.066926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:39.067604 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.067109 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:39.067604 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.067184 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs podName:379b2ddd-c7ae-4cda-ab35-996636b9eb3c nodeName:}" failed. No retries permitted until 2026-07-22 04:34:11.06716268 +0000 UTC m=+66.115430931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs") pod "network-metrics-daemon-2bwfx" (UID: "379b2ddd-c7ae-4cda-ab35-996636b9eb3c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:33:39.167984 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.167957 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:39.168158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.168017 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:39.168158 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168133 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:39.168263 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168198 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret podName:a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:11.168181348 +0000 UTC m=+66.216449602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret") pod "global-pull-secret-syncer-95d2d" (UID: "a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:33:39.168263 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168135 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:33:39.168263 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168225 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:33:39.168263 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168234 2555 projected.go:196] Error preparing data for projected volume kube-api-access-hvt5m for pod openshift-network-diagnostics/network-check-target-m2gjh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:39.168444 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.168262 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m podName:284e065b-de22-4f10-b3a5-a17d61717871 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:11.16825526 +0000 UTC m=+66.216523497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hvt5m" (UniqueName: "kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m") pod "network-check-target-m2gjh" (UID: "284e065b-de22-4f10-b3a5-a17d61717871") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:33:39.242696 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.242668 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-75.ec2.internal" event="NodeReady" Jul 22 04:33:39.242838 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.242792 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 04:33:39.291110 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.291081 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:33:39.294990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.294971 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b6c5q"] Jul 22 04:33:39.295119 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.295108 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.297646 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.297625 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 04:33:39.297790 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.297673 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 04:33:39.297891 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.297870 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 04:33:39.298002 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.297901 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9jfd\"" Jul 22 04:33:39.298240 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.298226 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pxjdd"] Jul 22 04:33:39.298370 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.298353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.301146 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.301116 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pmc57\"" Jul 22 04:33:39.301146 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.301141 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 04:33:39.301741 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.301497 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.301741 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.301556 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 04:33:39.301741 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.301648 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 04:33:39.303521 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.303501 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 04:33:39.304448 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.303715 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 04:33:39.304448 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.303803 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 04:33:39.305314 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.305295 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hhmhx\"" Jul 22 04:33:39.305898 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.305874 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b6c5q"] Jul 22 04:33:39.309526 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.309442 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pxjdd"] Jul 22 04:33:39.310055 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.309881 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:33:39.424576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.424541 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:33:39.424576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.424563 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:33:39.424815 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.424733 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:33:39.428062 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.427158 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 04:33:39.428062 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.427834 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ljtjg\"" Jul 22 04:33:39.428062 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.427902 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 04:33:39.428278 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.428134 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 04:33:39.428278 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.428173 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j88w2\"" Jul 22 04:33:39.428948 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.428439 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 04:33:39.470457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470433 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.470563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470475 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470507 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-cert\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.470563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470541 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470681 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470633 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470681 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470664 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56ndq\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470762 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470694 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7gp7\" (UniqueName: \"kubernetes.io/projected/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-kube-api-access-f7gp7\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.470762 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470722 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/618eb713-6cb2-46cc-89aa-2e84b3eb848b-config-volume\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.470762 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470746 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470862 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470784 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/618eb713-6cb2-46cc-89aa-2e84b3eb848b-tmp-dir\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.470862 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470878 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.470958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470908 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn92q\" (UniqueName: \"kubernetes.io/projected/618eb713-6cb2-46cc-89aa-2e84b3eb848b-kube-api-access-jn92q\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.470958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.470938 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572301 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572228 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn92q\" (UniqueName: \"kubernetes.io/projected/618eb713-6cb2-46cc-89aa-2e84b3eb848b-kube-api-access-jn92q\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.572301 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572301 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572296 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.572395 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572417 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.572457 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:33:40.072436239 +0000 UTC m=+35.120704476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.572398 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.572482 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:39.572542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572487 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-cert\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572547 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572591 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572617 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56ndq\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572664 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7gp7\" (UniqueName: \"kubernetes.io/projected/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-kube-api-access-f7gp7\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:39.572688 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:40.072668498 +0000 UTC m=+35.120936737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/618eb713-6cb2-46cc-89aa-2e84b3eb848b-config-volume\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.572805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572761 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.573144 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572815 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/618eb713-6cb2-46cc-89aa-2e84b3eb848b-tmp-dir\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.573144 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572848 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.573144 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.572877 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.573354 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.573333 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.573556 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.573518 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.573751 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.573727 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/618eb713-6cb2-46cc-89aa-2e84b3eb848b-tmp-dir\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.574046 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.573889 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/618eb713-6cb2-46cc-89aa-2e84b3eb848b-config-volume\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.574267 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.574250 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.577678 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.577657 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.577678 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.577668 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.577904 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.577880 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-cert\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.581487 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.581457 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn92q\" (UniqueName: \"kubernetes.io/projected/618eb713-6cb2-46cc-89aa-2e84b3eb848b-kube-api-access-jn92q\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:39.582036 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.581993 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.582304 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.582276 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7gp7\" (UniqueName: \"kubernetes.io/projected/13b2bd6f-8ed7-4e1f-9920-5dba9a915009-kube-api-access-f7gp7\") pod \"ingress-canary-b6c5q\" (UID: \"13b2bd6f-8ed7-4e1f-9920-5dba9a915009\") " pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:39.582474 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.582408 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56ndq\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:39.616792 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:39.616764 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b6c5q" Jul 22 04:33:40.077645 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:40.077607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:40.077746 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:40.077748 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:40.077797 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:40.077816 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:40.077851 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:33:41.077828376 +0000 UTC m=+36.126096618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:40.078092 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:40.077878 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:41.07786557 +0000 UTC m=+36.126133808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:40.855611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:40.855485 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b6c5q"] Jul 22 04:33:40.858803 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:33:40.858776 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b2bd6f_8ed7_4e1f_9920_5dba9a915009.slice/crio-a28fdb5cc53028d9708cca681999434a289aeddc034e9a4274233b5e59536530 WatchSource:0}: Error finding container a28fdb5cc53028d9708cca681999434a289aeddc034e9a4274233b5e59536530: Status 404 returned error can't find the container with id a28fdb5cc53028d9708cca681999434a289aeddc034e9a4274233b5e59536530 Jul 22 04:33:41.085768 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:41.085706 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:41.085768 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:41.085748 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:41.086103 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:41.085848 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:41.086103 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:41.085926 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:33:43.085909873 +0000 UTC m=+38.134178126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:41.086103 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:41.085853 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:41.086103 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:41.085950 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:41.086103 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:41.085999 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:43.085984971 +0000 UTC m=+38.134253222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:41.564047 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:41.563939 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="e4517b89cb08e122b22d7fc9e96cb9acf12a5c68d305e67a2e71bc47fd6f6451" exitCode=0 Jul 22 04:33:41.564197 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:41.564040 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"e4517b89cb08e122b22d7fc9e96cb9acf12a5c68d305e67a2e71bc47fd6f6451"} Jul 22 04:33:41.565408 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:41.565385 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b6c5q" event={"ID":"13b2bd6f-8ed7-4e1f-9920-5dba9a915009","Type":"ContainerStarted","Data":"a28fdb5cc53028d9708cca681999434a289aeddc034e9a4274233b5e59536530"} Jul 22 04:33:42.571561 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:42.571523 2555 generic.go:356] "Generic (PLEG): container finished" podID="b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4" containerID="9c0cbf6348dcfd383a831e7094633665a59ba090f43f4b723c48e9ca0cdb9c61" exitCode=0 Jul 22 04:33:42.572086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:42.571570 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerDied","Data":"9c0cbf6348dcfd383a831e7094633665a59ba090f43f4b723c48e9ca0cdb9c61"} Jul 22 04:33:43.103548 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.103446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:43.103548 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.103490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:43.103807 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:43.103595 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:43.103807 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:43.103613 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:43.103807 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:43.103637 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:43.103807 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:43.103645 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:33:47.103632676 +0000 UTC m=+42.151900913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:43.103807 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:43.103689 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:47.103673094 +0000 UTC m=+42.151941345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:43.577539 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.577449 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" event={"ID":"b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4","Type":"ContainerStarted","Data":"23db357865d1689f52be587ff79315f515424ca5849443de3ede96df6917139e"} Jul 22 04:33:43.578990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.578963 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b6c5q" event={"ID":"13b2bd6f-8ed7-4e1f-9920-5dba9a915009","Type":"ContainerStarted","Data":"932c5db22898efc90b0201d937d898975d4add849f86c5215dbad0ce2ab5e691"} Jul 22 04:33:43.599233 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.599182 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nzzlb" podStartSLOduration=8.638879562 podStartE2EDuration="37.59916568s" totalImagesPullingTime="28.960286118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:43.597400843 +0000 UTC m=+38.645669102" watchObservedRunningTime="2026-07-22 04:33:43.59916568 +0000 UTC m=+38.647433939" Jul 22 04:33:43.612130 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:43.612084 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b6c5q" podStartSLOduration=2.558935506 podStartE2EDuration="4.612068796s" totalImagesPullingTime="2.05313329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:43.611469337 +0000 UTC m=+38.659737595" watchObservedRunningTime="2026-07-22 04:33:43.612068796 +0000 UTC m=+38.660337033" Jul 22 04:33:47.132453 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:47.132415 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:47.132453 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:47.132457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:47.132950 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:47.132565 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:47.132950 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:47.132620 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:33:55.132606292 +0000 UTC m=+50.180874529 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:47.132950 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:47.132569 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:47.132950 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:47.132660 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:47.132950 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:47.132703 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:33:55.132691746 +0000 UTC m=+50.180960012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:50.767015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.766983 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmm6"] Jul 22 04:33:50.795219 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.795189 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmm6"] Jul 22 04:33:50.795351 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.795305 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.798177 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.798151 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 04:33:50.798283 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.798162 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 04:33:50.798339 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.798279 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 04:33:50.798416 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.798403 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tc9bh\"" Jul 22 04:33:50.799353 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.799338 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 04:33:50.854595 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.854574 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-data-volume\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.854705 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.854603 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.854705 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.854636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-crio-socket\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.854779 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.854755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gms6\" (UniqueName: \"kubernetes.io/projected/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-api-access-6gms6\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.854812 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.854789 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955211 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955181 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gms6\" (UniqueName: \"kubernetes.io/projected/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-api-access-6gms6\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955352 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955352 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955246 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-data-volume\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955352 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955264 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955352 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-crio-socket\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955565 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:50.955355 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:33:50.955565 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:50.955402 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:33:50.955565 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:50.955424 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:51.455405886 +0000 UTC m=+46.503674125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : secret "insights-runtime-extractor-tls" not found Jul 22 04:33:50.955565 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:50.955458 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:51.455442287 +0000 UTC m=+46.503710538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : configmap "kube-rbac-proxy" not found Jul 22 04:33:50.955565 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955498 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-crio-socket\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.955747 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.955571 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-data-volume\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:50.964583 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:50.964561 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gms6\" (UniqueName: \"kubernetes.io/projected/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-api-access-6gms6\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:51.458019 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:51.457985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:51.458197 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:51.458042 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:51.458197 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:51.458135 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:33:51.458197 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:51.458161 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:33:51.458302 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:51.458205 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:52.458189024 +0000 UTC m=+47.506457263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : secret "insights-runtime-extractor-tls" not found Jul 22 04:33:51.458302 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:51.458218 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:52.458212317 +0000 UTC m=+47.506480554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : configmap "kube-rbac-proxy" not found Jul 22 04:33:52.465454 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:52.465421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:52.465454 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:52.465466 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:52.465869 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:52.465571 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:33:52.465869 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:52.465587 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:33:52.465869 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:52.465623 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:54.465610318 +0000 UTC m=+49.513878568 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : configmap "kube-rbac-proxy" not found Jul 22 04:33:52.465869 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:52.465643 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:54.465631072 +0000 UTC m=+49.513899310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : secret "insights-runtime-extractor-tls" not found Jul 22 04:33:52.573477 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:52.573453 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7266c_4cd9108c-5501-48c3-a5ea-879f5e0a727f/dns-node-resolver/0.log" Jul 22 04:33:53.974772 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:53.974745 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wfvlw_64d5801c-fe21-44dd-af4e-de832e04007c/node-ca/0.log" Jul 22 04:33:54.375766 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:54.375732 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b6c5q_13b2bd6f-8ed7-4e1f-9920-5dba9a915009/serve-healthcheck-canary/0.log" Jul 22 04:33:54.476708 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:54.476663 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:54.476708 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:54.476715 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:54.476938 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:54.476822 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:33:54.476938 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:54.476881 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:58.476866376 +0000 UTC m=+53.525134612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : secret "insights-runtime-extractor-tls" not found Jul 22 04:33:54.476938 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:54.476909 2555 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:33:54.477109 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:54.476968 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:33:58.476956418 +0000 UTC m=+53.525224655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : configmap "kube-rbac-proxy" not found Jul 22 04:33:55.181140 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:55.181104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:55.181152 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:55.181252 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:55.181275 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6674bbbc96-lgzxz: secret "image-registry-tls" not found Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:55.181319 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:55.181330 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls podName:f145647b-e904-4020-a03f-533c7710b1e8 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:11.181314483 +0000 UTC m=+66.229582720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls") pod "image-registry-6674bbbc96-lgzxz" (UID: "f145647b-e904-4020-a03f-533c7710b1e8") : secret "image-registry-tls" not found Jul 22 04:33:55.181557 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:55.181370 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls podName:618eb713-6cb2-46cc-89aa-2e84b3eb848b nodeName:}" failed. No retries permitted until 2026-07-22 04:34:11.181353549 +0000 UTC m=+66.229621793 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls") pod "dns-default-pxjdd" (UID: "618eb713-6cb2-46cc-89aa-2e84b3eb848b") : secret "dns-default-metrics-tls" not found Jul 22 04:33:58.509129 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:58.509088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:58.509560 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:58.509141 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:33:58.509560 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:58.509237 2555 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:33:58.509560 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:33:58.509298 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls podName:80f60688-875a-4d8a-8cf7-9f7f1eabaf7a nodeName:}" failed. No retries permitted until 2026-07-22 04:34:06.509283845 +0000 UTC m=+61.557552082 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mvmm6" (UID: "80f60688-875a-4d8a-8cf7-9f7f1eabaf7a") : secret "insights-runtime-extractor-tls" not found Jul 22 04:33:58.509669 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:33:58.509604 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:34:04.560939 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:04.560910 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5752d" Jul 22 04:34:06.570248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:06.570213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:34:06.572466 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:06.572431 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/80f60688-875a-4d8a-8cf7-9f7f1eabaf7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mvmm6\" (UID: \"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a\") " pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:34:06.707742 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:06.707720 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tc9bh\"" Jul 22 04:34:06.715750 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:06.715733 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mvmm6" Jul 22 04:34:06.833087 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:06.833008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mvmm6"] Jul 22 04:34:06.835918 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:06.835895 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f60688_875a_4d8a_8cf7_9f7f1eabaf7a.slice/crio-ddce19f23ecd27d97cb29360ebc23357c1331e191c7a1f8301283a6eccfec90d WatchSource:0}: Error finding container ddce19f23ecd27d97cb29360ebc23357c1331e191c7a1f8301283a6eccfec90d: Status 404 returned error can't find the container with id ddce19f23ecd27d97cb29360ebc23357c1331e191c7a1f8301283a6eccfec90d Jul 22 04:34:07.636617 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:07.636570 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmm6" event={"ID":"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a","Type":"ContainerStarted","Data":"ddce19f23ecd27d97cb29360ebc23357c1331e191c7a1f8301283a6eccfec90d"} Jul 22 04:34:11.106755 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.106714 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:34:11.109599 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.109574 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 04:34:11.119608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.119588 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/379b2ddd-c7ae-4cda-ab35-996636b9eb3c-metrics-certs\") pod \"network-metrics-daemon-2bwfx\" (UID: \"379b2ddd-c7ae-4cda-ab35-996636b9eb3c\") " pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:34:11.207082 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.207055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:34:11.207224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.207115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:11.207224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.207145 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:34:11.207224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.207187 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:34:11.209646 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.209618 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 04:34:11.209770 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.209618 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 04:34:11.209770 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.209754 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"image-registry-6674bbbc96-lgzxz\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:11.210805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.210779 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/618eb713-6cb2-46cc-89aa-2e84b3eb848b-metrics-tls\") pod \"dns-default-pxjdd\" (UID: \"618eb713-6cb2-46cc-89aa-2e84b3eb848b\") " pod="openshift-dns/dns-default-pxjdd" Jul 22 04:34:11.219692 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.219669 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8-original-pull-secret\") pod \"global-pull-secret-syncer-95d2d\" (UID: \"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8\") " pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:34:11.219795 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.219755 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 04:34:11.231064 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.231038 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvt5m\" (UniqueName: \"kubernetes.io/projected/284e065b-de22-4f10-b3a5-a17d61717871-kube-api-access-hvt5m\") pod \"network-check-target-m2gjh\" (UID: \"284e065b-de22-4f10-b3a5-a17d61717871\") " pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:34:11.238191 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.238171 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-95d2d" Jul 22 04:34:11.247299 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.247281 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j88w2\"" Jul 22 04:34:11.252115 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.252098 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ljtjg\"" Jul 22 04:34:11.255921 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.255902 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:34:11.260630 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.260607 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bwfx" Jul 22 04:34:11.410978 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.410898 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9jfd\"" Jul 22 04:34:11.418456 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.418434 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:11.425760 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.425734 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hhmhx\"" Jul 22 04:34:11.434442 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.434427 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pxjdd" Jul 22 04:34:11.671538 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.671425 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-95d2d"] Jul 22 04:34:11.676335 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:11.676308 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b3084d_a1cd_481b_abd1_c4cd3d9dd3c8.slice/crio-342368a0684ea471b91c21d5885d73c5fedad3ae15fdef37a07aa3393ed8fea8 WatchSource:0}: Error finding container 342368a0684ea471b91c21d5885d73c5fedad3ae15fdef37a07aa3393ed8fea8: Status 404 returned error can't find the container with id 342368a0684ea471b91c21d5885d73c5fedad3ae15fdef37a07aa3393ed8fea8 Jul 22 04:34:11.689053 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.688928 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m2gjh"] Jul 22 04:34:11.691965 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:11.691916 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod284e065b_de22_4f10_b3a5_a17d61717871.slice/crio-968777130e54a152c6234b8a2b39349b6f1eec8407320282ce4d17f9fc14d6da WatchSource:0}: Error finding container 968777130e54a152c6234b8a2b39349b6f1eec8407320282ce4d17f9fc14d6da: Status 404 returned error can't find the container with id 968777130e54a152c6234b8a2b39349b6f1eec8407320282ce4d17f9fc14d6da Jul 22 04:34:11.917159 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.917128 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bwfx"] Jul 22 04:34:11.920245 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:11.920221 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379b2ddd_c7ae_4cda_ab35_996636b9eb3c.slice/crio-cb45f0f2279e8512e87a110bad97a18a82a75da0325ffdd5f72254b382b15105 WatchSource:0}: Error finding container cb45f0f2279e8512e87a110bad97a18a82a75da0325ffdd5f72254b382b15105: Status 404 returned error can't find the container with id cb45f0f2279e8512e87a110bad97a18a82a75da0325ffdd5f72254b382b15105 Jul 22 04:34:11.920350 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.920282 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pxjdd"] Jul 22 04:34:11.921359 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:11.921284 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:34:11.924870 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:11.924850 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618eb713_6cb2_46cc_89aa_2e84b3eb848b.slice/crio-d9f24765ed9ea0a862010b526c5527007959ab733ae8b0609e92d49ce386af07 WatchSource:0}: Error finding container d9f24765ed9ea0a862010b526c5527007959ab733ae8b0609e92d49ce386af07: Status 404 returned error can't find the container with id d9f24765ed9ea0a862010b526c5527007959ab733ae8b0609e92d49ce386af07 Jul 22 04:34:11.925437 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:11.925420 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf145647b_e904_4020_a03f_533c7710b1e8.slice/crio-d6c3ca758f1fdf95b801bef2a814535d6c81a374795cd4745eb001c04af31a8f WatchSource:0}: Error finding container d6c3ca758f1fdf95b801bef2a814535d6c81a374795cd4745eb001c04af31a8f: Status 404 returned error can't find the container with id d6c3ca758f1fdf95b801bef2a814535d6c81a374795cd4745eb001c04af31a8f Jul 22 04:34:12.652590 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.652521 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmm6" event={"ID":"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a","Type":"ContainerStarted","Data":"93840913cbbd5bef57c853433e56350c1ec16e538033eb379c427810916603cf"} Jul 22 04:34:12.654319 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.654282 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bwfx" event={"ID":"379b2ddd-c7ae-4cda-ab35-996636b9eb3c","Type":"ContainerStarted","Data":"cb45f0f2279e8512e87a110bad97a18a82a75da0325ffdd5f72254b382b15105"} Jul 22 04:34:12.656263 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.656089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" event={"ID":"f145647b-e904-4020-a03f-533c7710b1e8","Type":"ContainerStarted","Data":"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d"} Jul 22 04:34:12.656263 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.656134 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" event={"ID":"f145647b-e904-4020-a03f-533c7710b1e8","Type":"ContainerStarted","Data":"d6c3ca758f1fdf95b801bef2a814535d6c81a374795cd4745eb001c04af31a8f"} Jul 22 04:34:12.656263 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.656217 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:12.657686 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.657635 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m2gjh" event={"ID":"284e065b-de22-4f10-b3a5-a17d61717871","Type":"ContainerStarted","Data":"968777130e54a152c6234b8a2b39349b6f1eec8407320282ce4d17f9fc14d6da"} Jul 22 04:34:12.659104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.659083 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-95d2d" event={"ID":"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8","Type":"ContainerStarted","Data":"342368a0684ea471b91c21d5885d73c5fedad3ae15fdef37a07aa3393ed8fea8"} Jul 22 04:34:12.660586 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:12.660555 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxjdd" event={"ID":"618eb713-6cb2-46cc-89aa-2e84b3eb848b","Type":"ContainerStarted","Data":"d9f24765ed9ea0a862010b526c5527007959ab733ae8b0609e92d49ce386af07"} Jul 22 04:34:15.055713 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.055651 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" podStartSLOduration=46.055627442 podStartE2EDuration="46.055627442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:12.675903258 +0000 UTC m=+67.724171520" watchObservedRunningTime="2026-07-22 04:34:15.055627442 +0000 UTC m=+70.103895703" Jul 22 04:34:15.056543 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.056509 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58"] Jul 22 04:34:15.058393 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.058373 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs"] Jul 22 04:34:15.058521 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.058501 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.060279 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.060262 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.061343 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.061323 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 04:34:15.061440 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.061323 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 04:34:15.061440 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.061371 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 04:34:15.062447 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.062428 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 04:34:15.062587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.062568 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 04:34:15.062661 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.062640 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-frp6w\"" Jul 22 04:34:15.072070 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.072008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs"] Jul 22 04:34:15.073057 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.072987 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58"] Jul 22 04:34:15.116634 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.116606 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:34:15.133392 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.133365 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/82f74c97-a149-42b8-a578-a979c50fb5cf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.133535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.133416 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99586285-acef-47f8-8e16-394717cf2ea6-tmp\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.133535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.133466 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99586285-acef-47f8-8e16-394717cf2ea6-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.133535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.133490 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb4l9\" (UniqueName: \"kubernetes.io/projected/99586285-acef-47f8-8e16-394717cf2ea6-kube-api-access-vb4l9\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.133535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.133518 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbkkd\" (UniqueName: \"kubernetes.io/projected/82f74c97-a149-42b8-a578-a979c50fb5cf-kube-api-access-rbkkd\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.178423 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.178394 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dd6957979-g82wp"] Jul 22 04:34:15.180736 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.180709 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.195086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.195064 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd6957979-g82wp"] Jul 22 04:34:15.234512 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-installation-pull-secrets\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.234668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234514 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-bound-sa-token\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.234668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234550 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99586285-acef-47f8-8e16-394717cf2ea6-tmp\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.234668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234626 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbpw9\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-kube-api-access-fbpw9\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.234668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234653 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-registry-tls\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.234941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234677 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbkkd\" (UniqueName: \"kubernetes.io/projected/82f74c97-a149-42b8-a578-a979c50fb5cf-kube-api-access-rbkkd\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.234941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234717 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/82f74c97-a149-42b8-a578-a979c50fb5cf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.234941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234876 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-registry-certificates\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.234941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234884 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99586285-acef-47f8-8e16-394717cf2ea6-tmp\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.234941 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234939 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-trusted-ca\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.235227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.234974 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99586285-acef-47f8-8e16-394717cf2ea6-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.235227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.235003 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-image-registry-private-configuration\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.235227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.235051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb4l9\" (UniqueName: \"kubernetes.io/projected/99586285-acef-47f8-8e16-394717cf2ea6-kube-api-access-vb4l9\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.235227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.235103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3127d883-9f15-4947-857d-47e76ff4d463-ca-trust-extracted\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.237506 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.237481 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/99586285-acef-47f8-8e16-394717cf2ea6-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.237609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.237570 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/82f74c97-a149-42b8-a578-a979c50fb5cf-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.243429 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.243369 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbkkd\" (UniqueName: \"kubernetes.io/projected/82f74c97-a149-42b8-a578-a979c50fb5cf-kube-api-access-rbkkd\") pod \"managed-serviceaccount-addon-agent-648b548986-d45zs\" (UID: \"82f74c97-a149-42b8-a578-a979c50fb5cf\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.243530 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.243495 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb4l9\" (UniqueName: \"kubernetes.io/projected/99586285-acef-47f8-8e16-394717cf2ea6-kube-api-access-vb4l9\") pod \"klusterlet-addon-workmgr-67c98fb7b6-rnc58\" (UID: \"99586285-acef-47f8-8e16-394717cf2ea6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.336121 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-image-registry-private-configuration\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336121 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336095 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3127d883-9f15-4947-857d-47e76ff4d463-ca-trust-extracted\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336311 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-installation-pull-secrets\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336311 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336232 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-bound-sa-token\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336359 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbpw9\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-kube-api-access-fbpw9\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336403 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-registry-tls\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336517 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336457 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-registry-certificates\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336517 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336486 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-trusted-ca\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.336790 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.336737 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3127d883-9f15-4947-857d-47e76ff4d463-ca-trust-extracted\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.337474 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.337446 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-registry-certificates\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.337595 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.337463 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3127d883-9f15-4947-857d-47e76ff4d463-trusted-ca\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.339036 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.338973 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-installation-pull-secrets\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.339036 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.338991 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3127d883-9f15-4947-857d-47e76ff4d463-image-registry-private-configuration\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.339248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.339226 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-registry-tls\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.346169 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.346126 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbpw9\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-kube-api-access-fbpw9\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.346672 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.346632 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3127d883-9f15-4947-857d-47e76ff4d463-bound-sa-token\") pod \"image-registry-6dd6957979-g82wp\" (UID: \"3127d883-9f15-4947-857d-47e76ff4d463\") " pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:15.370045 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.370003 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:15.383079 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.383057 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" Jul 22 04:34:15.490451 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:15.490417 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:19.132503 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.128171 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58"] Jul 22 04:34:19.147420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.147148 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs"] Jul 22 04:34:19.150419 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:19.150390 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82f74c97_a149_42b8_a578_a979c50fb5cf.slice/crio-d6cb6b022a2287e718582d3c8aa069dc9dfd2b6ccce6673c14f7c9a444443ffe WatchSource:0}: Error finding container d6cb6b022a2287e718582d3c8aa069dc9dfd2b6ccce6673c14f7c9a444443ffe: Status 404 returned error can't find the container with id d6cb6b022a2287e718582d3c8aa069dc9dfd2b6ccce6673c14f7c9a444443ffe Jul 22 04:34:19.187214 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.186876 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dd6957979-g82wp"] Jul 22 04:34:19.192869 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:19.192833 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3127d883_9f15_4947_857d_47e76ff4d463.slice/crio-efe9761b925bfaca2a8c2e9f4aef299a0b50e92dbbb5c266320049234c25a39e WatchSource:0}: Error finding container efe9761b925bfaca2a8c2e9f4aef299a0b50e92dbbb5c266320049234c25a39e: Status 404 returned error can't find the container with id efe9761b925bfaca2a8c2e9f4aef299a0b50e92dbbb5c266320049234c25a39e Jul 22 04:34:19.276629 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.276598 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm"] Jul 22 04:34:19.279526 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.279499 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:19.282073 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.282048 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 04:34:19.282185 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.282121 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mfh2q\"" Jul 22 04:34:19.288899 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.288828 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm"] Jul 22 04:34:19.368683 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.368654 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-jhtkm\" (UID: \"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:19.469231 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.469151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-jhtkm\" (UID: \"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:19.469367 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:19.469295 2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 22 04:34:19.469407 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:19.469376 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates podName:b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:19.969353778 +0000 UTC m=+75.017622030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-jhtkm" (UID: "b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4") : secret "prometheus-operator-admission-webhook-tls" not found Jul 22 04:34:19.686364 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.686324 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmm6" event={"ID":"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a","Type":"ContainerStarted","Data":"b40b6b6d62997c31b5257d1662338da95225ee476fb82029c27ece34b5bcff0d"} Jul 22 04:34:19.688284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.688257 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bwfx" event={"ID":"379b2ddd-c7ae-4cda-ab35-996636b9eb3c","Type":"ContainerStarted","Data":"3d155c9527aeb551ce043aaf5b49b1f69cb34cb83dacf789cf9d6921cd75aa0d"} Jul 22 04:34:19.688417 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.688309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bwfx" event={"ID":"379b2ddd-c7ae-4cda-ab35-996636b9eb3c","Type":"ContainerStarted","Data":"086eb8186fa93029d3426787823e47c2bde3ff3a392cd118038d066e4977d255"} Jul 22 04:34:19.689974 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.689949 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m2gjh" event={"ID":"284e065b-de22-4f10-b3a5-a17d61717871","Type":"ContainerStarted","Data":"505df0a20f2c174d8904afe4e87705bd2ee19a16c9f4c6d1aa6d3c1568fe24e0"} Jul 22 04:34:19.690140 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.690000 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:34:19.692292 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.692258 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" event={"ID":"3127d883-9f15-4947-857d-47e76ff4d463","Type":"ContainerStarted","Data":"d776b686434378cbc62de12ba22ae9b455a0cfe03e5e529c794782052c97a3f9"} Jul 22 04:34:19.692406 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.692306 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" event={"ID":"3127d883-9f15-4947-857d-47e76ff4d463","Type":"ContainerStarted","Data":"efe9761b925bfaca2a8c2e9f4aef299a0b50e92dbbb5c266320049234c25a39e"} Jul 22 04:34:19.692469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.692428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:19.694717 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.694691 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" event={"ID":"82f74c97-a149-42b8-a578-a979c50fb5cf","Type":"ContainerStarted","Data":"d6cb6b022a2287e718582d3c8aa069dc9dfd2b6ccce6673c14f7c9a444443ffe"} Jul 22 04:34:19.696719 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.696696 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" event={"ID":"99586285-acef-47f8-8e16-394717cf2ea6","Type":"ContainerStarted","Data":"6dfd4c342eb0e39755fa94160cbad027d820c87c5ecfcd474d4114eb1c13ef20"} Jul 22 04:34:19.698258 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.698230 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-95d2d" event={"ID":"a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8","Type":"ContainerStarted","Data":"f7a4d83b2696784534d3a0b2eddd05b70e771348df9870ecafb62f1f323bfd95"} Jul 22 04:34:19.699973 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.699955 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxjdd" event={"ID":"618eb713-6cb2-46cc-89aa-2e84b3eb848b","Type":"ContainerStarted","Data":"145efd8895a7f87e641c08e310281c41c3eaaa5f1fe2442852c6e6d1b52b83af"} Jul 22 04:34:19.700071 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.699989 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pxjdd" event={"ID":"618eb713-6cb2-46cc-89aa-2e84b3eb848b","Type":"ContainerStarted","Data":"3406383b1c1beace0e0ccf8aa2347d65a76ded8f99a5e960a105fe39db886eee"} Jul 22 04:34:19.700138 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.700080 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pxjdd" Jul 22 04:34:19.703791 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.703753 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2bwfx" podStartSLOduration=66.757049882 podStartE2EDuration="1m13.70374021s" totalImagesPullingTime="6.946690328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:19.702650411 +0000 UTC m=+74.750918682" watchObservedRunningTime="2026-07-22 04:34:19.70374021 +0000 UTC m=+74.752008480" Jul 22 04:34:19.719349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.719274 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pxjdd" podStartSLOduration=33.697431693 podStartE2EDuration="40.719258649s" totalImagesPullingTime="7.021826956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:19.718245294 +0000 UTC m=+74.766513554" watchObservedRunningTime="2026-07-22 04:34:19.719258649 +0000 UTC m=+74.767526907" Jul 22 04:34:19.735298 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.735170 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" podStartSLOduration=4.735153074 podStartE2EDuration="4.735153074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:19.734862909 +0000 UTC m=+74.783131169" watchObservedRunningTime="2026-07-22 04:34:19.735153074 +0000 UTC m=+74.783421334" Jul 22 04:34:19.750009 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.749506 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-95d2d" podStartSLOduration=66.475768236 podStartE2EDuration="1m13.749494394s" totalImagesPullingTime="7.273726158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:19.748919186 +0000 UTC m=+74.797187446" watchObservedRunningTime="2026-07-22 04:34:19.749494394 +0000 UTC m=+74.797762653" Jul 22 04:34:19.764228 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.764148 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m2gjh" podStartSLOduration=67.505957266 podStartE2EDuration="1m14.764126264s" totalImagesPullingTime="7.258168998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:19.762989076 +0000 UTC m=+74.811257316" watchObservedRunningTime="2026-07-22 04:34:19.764126264 +0000 UTC m=+74.812394525" Jul 22 04:34:19.973499 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:19.973414 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-jhtkm\" (UID: \"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:19.973631 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:19.973548 2555 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 22 04:34:19.973631 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:19.973614 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates podName:b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:20.973594672 +0000 UTC m=+76.021862912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-jhtkm" (UID: "b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4") : secret "prometheus-operator-admission-webhook-tls" not found Jul 22 04:34:20.984519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:20.984475 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-jhtkm\" (UID: \"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:20.988860 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:20.988797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-jhtkm\" (UID: \"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:21.092310 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:21.092276 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:24.519100 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.518717 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm"] Jul 22 04:34:24.522013 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:24.521989 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22eef00_8b14_4fdc_a5a9_e2f86d4a03e4.slice/crio-0c4c9b7d726141bccad71faac91d6ff9aab21b70ec19818c97ab1cae5993aa40 WatchSource:0}: Error finding container 0c4c9b7d726141bccad71faac91d6ff9aab21b70ec19818c97ab1cae5993aa40: Status 404 returned error can't find the container with id 0c4c9b7d726141bccad71faac91d6ff9aab21b70ec19818c97ab1cae5993aa40 Jul 22 04:34:24.725329 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.725255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" event={"ID":"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4","Type":"ContainerStarted","Data":"0c4c9b7d726141bccad71faac91d6ff9aab21b70ec19818c97ab1cae5993aa40"} Jul 22 04:34:24.727222 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.727195 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mvmm6" event={"ID":"80f60688-875a-4d8a-8cf7-9f7f1eabaf7a","Type":"ContainerStarted","Data":"d6fa9d7c888a35b100798f9fd020888d127b5bd34cca3417a49d7d66fe5edb1c"} Jul 22 04:34:24.728620 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.728592 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" event={"ID":"99586285-acef-47f8-8e16-394717cf2ea6","Type":"ContainerStarted","Data":"781de000b235fd6888e821497da0a7b12af89aa3950a91ad4737d5417fa386d1"} Jul 22 04:34:24.728800 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.728784 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:24.730505 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.730485 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" Jul 22 04:34:24.744746 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.744706 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mvmm6" podStartSLOduration=17.420576622 podStartE2EDuration="34.744692335s" totalImagesPullingTime="17.324115713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:33:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:24.744042068 +0000 UTC m=+79.792310328" watchObservedRunningTime="2026-07-22 04:34:24.744692335 +0000 UTC m=+79.792960593" Jul 22 04:34:24.758959 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:24.758867 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c98fb7b6-rnc58" podStartSLOduration=4.490222965 podStartE2EDuration="9.758850157s" totalImagesPullingTime="5.268627192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:24.758237694 +0000 UTC m=+79.806505953" watchObservedRunningTime="2026-07-22 04:34:24.758850157 +0000 UTC m=+79.807118419" Jul 22 04:34:25.733978 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:25.733948 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" event={"ID":"b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4","Type":"ContainerStarted","Data":"3c9bb9168cb66f81d96ad277b333a3d2859348a114b5de53dd3e1cf877f44a2f"} Jul 22 04:34:25.734399 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:25.734379 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:25.738974 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:25.738949 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" Jul 22 04:34:25.762224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:25.762182 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-jhtkm" podStartSLOduration=5.648322598 podStartE2EDuration="6.762169963s" totalImagesPullingTime="1.113847365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:25.747804703 +0000 UTC m=+80.796072968" watchObservedRunningTime="2026-07-22 04:34:25.762169963 +0000 UTC m=+80.810438199" Jul 22 04:34:29.708911 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:29.708778 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pxjdd" Jul 22 04:34:30.726892 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.726852 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vv7b8"] Jul 22 04:34:30.731856 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.731835 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fw8ht"] Jul 22 04:34:30.731979 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.731963 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.735191 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.735167 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.736099 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736078 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 04:34:30.736223 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736198 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 04:34:30.736275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736261 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 04:34:30.736350 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736335 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 04:34:30.736426 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736407 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 04:34:30.736572 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.736552 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 04:34:30.737166 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.737141 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-b85zg\"" Jul 22 04:34:30.737643 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.737624 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 04:34:30.737743 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.737683 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 04:34:30.737743 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.737719 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zm9xl\"" Jul 22 04:34:30.737856 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.737783 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 04:34:30.744740 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.744718 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vv7b8"] Jul 22 04:34:30.850484 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850461 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-textfile\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850487 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-sys\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850504 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bs2b\" (UniqueName: \"kubernetes.io/projected/c612385a-5af9-4b1d-af28-e5e6ee949c96-kube-api-access-5bs2b\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850540 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad1b7570-747d-45bd-aaf8-ebc12c73c426-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850572 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850589 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850606 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79qfl\" (UniqueName: \"kubernetes.io/projected/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-api-access-79qfl\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-tls\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850658 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-accelerators-collector-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850684 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850707 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-root\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.850752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850753 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-wtmp\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.851096 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850768 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.851096 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.850793 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-metrics-client-ca\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951593 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad1b7570-747d-45bd-aaf8-ebc12c73c426-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951596 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951615 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79qfl\" (UniqueName: \"kubernetes.io/projected/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-api-access-79qfl\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-tls\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951667 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-accelerators-collector-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:30.951781 2555 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-root\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:30.951834 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls podName:ad1b7570-747d-45bd-aaf8-ebc12c73c426 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:31.451815664 +0000 UTC m=+86.500083918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-vv7b8" (UID: "ad1b7570-747d-45bd-aaf8-ebc12c73c426") : secret "kube-state-metrics-tls" not found Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951864 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951876 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-root\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951894 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-wtmp\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951921 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.951952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951953 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-metrics-client-ca\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951994 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-textfile\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.951996 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ad1b7570-747d-45bd-aaf8-ebc12c73c426-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952043 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-sys\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952071 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bs2b\" (UniqueName: \"kubernetes.io/projected/c612385a-5af9-4b1d-af28-e5e6ee949c96-kube-api-access-5bs2b\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-accelerators-collector-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952352 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-sys\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952380 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-wtmp\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952730 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952563 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.952730 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952566 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-textfile\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952730 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952709 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c612385a-5af9-4b1d-af28-e5e6ee949c96-metrics-client-ca\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.952730 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.952723 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.954379 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.954355 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:30.954379 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.954372 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-tls\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.954491 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.954462 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c612385a-5af9-4b1d-af28-e5e6ee949c96-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.959497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.959465 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bs2b\" (UniqueName: \"kubernetes.io/projected/c612385a-5af9-4b1d-af28-e5e6ee949c96-kube-api-access-5bs2b\") pod \"node-exporter-fw8ht\" (UID: \"c612385a-5af9-4b1d-af28-e5e6ee949c96\") " pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:30.959577 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:30.959538 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79qfl\" (UniqueName: \"kubernetes.io/projected/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-api-access-79qfl\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:31.049289 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.049264 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fw8ht" Jul 22 04:34:31.061647 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:31.061623 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc612385a_5af9_4b1d_af28_e5e6ee949c96.slice/crio-33b5812cebc4e15fca7ecebb343e910efa0b7279208cd9c28e1ea9b91742ee2e WatchSource:0}: Error finding container 33b5812cebc4e15fca7ecebb343e910efa0b7279208cd9c28e1ea9b91742ee2e: Status 404 returned error can't find the container with id 33b5812cebc4e15fca7ecebb343e910efa0b7279208cd9c28e1ea9b91742ee2e Jul 22 04:34:31.454323 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.454265 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:31.456393 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.456363 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad1b7570-747d-45bd-aaf8-ebc12c73c426-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-vv7b8\" (UID: \"ad1b7570-747d-45bd-aaf8-ebc12c73c426\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:31.652197 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.652145 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" Jul 22 04:34:31.759845 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.759809 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fw8ht" event={"ID":"c612385a-5af9-4b1d-af28-e5e6ee949c96","Type":"ContainerStarted","Data":"33b5812cebc4e15fca7ecebb343e910efa0b7279208cd9c28e1ea9b91742ee2e"} Jul 22 04:34:31.817786 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:31.817759 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-vv7b8"] Jul 22 04:34:31.935373 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:31.935341 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad1b7570_747d_45bd_aaf8_ebc12c73c426.slice/crio-d693b0c74a7d953a4ddbda9f572538ce820b01066386c3ccbc3a00f600cae57e WatchSource:0}: Error finding container d693b0c74a7d953a4ddbda9f572538ce820b01066386c3ccbc3a00f600cae57e: Status 404 returned error can't find the container with id d693b0c74a7d953a4ddbda9f572538ce820b01066386c3ccbc3a00f600cae57e Jul 22 04:34:32.764835 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:32.764798 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" event={"ID":"ad1b7570-747d-45bd-aaf8-ebc12c73c426","Type":"ContainerStarted","Data":"d693b0c74a7d953a4ddbda9f572538ce820b01066386c3ccbc3a00f600cae57e"} Jul 22 04:34:32.766385 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:32.766360 2555 generic.go:356] "Generic (PLEG): container finished" podID="c612385a-5af9-4b1d-af28-e5e6ee949c96" containerID="d1c62f319c10628db84ffe24e980034d566a71f94ceccb620891d3be45911855" exitCode=0 Jul 22 04:34:32.766508 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:32.766392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fw8ht" event={"ID":"c612385a-5af9-4b1d-af28-e5e6ee949c96","Type":"ContainerDied","Data":"d1c62f319c10628db84ffe24e980034d566a71f94ceccb620891d3be45911855"} Jul 22 04:34:33.688836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.688805 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5cf98d6887-f8bfj"] Jul 22 04:34:33.692438 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.692419 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.695102 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.694982 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 22 04:34:33.695102 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695065 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 22 04:34:33.695260 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695134 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-96617npnh9vrd\"" Jul 22 04:34:33.695260 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695153 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 22 04:34:33.695260 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695187 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 22 04:34:33.695260 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695254 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 22 04:34:33.695460 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.695297 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h78km\"" Jul 22 04:34:33.701931 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.701911 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cf98d6887-f8bfj"] Jul 22 04:34:33.772467 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.772434 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" event={"ID":"ad1b7570-747d-45bd-aaf8-ebc12c73c426","Type":"ContainerStarted","Data":"6c7fde2c121793091cbb51fa0a451db5da37b5feb3b544e4cb01e22d8861d436"} Jul 22 04:34:33.774778 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.774746 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fw8ht" event={"ID":"c612385a-5af9-4b1d-af28-e5e6ee949c96","Type":"ContainerStarted","Data":"5db0a73ef18fe54474b7536dcc654d11f56b012610abb78e59991679d22881fb"} Jul 22 04:34:33.774884 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.774795 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fw8ht" event={"ID":"c612385a-5af9-4b1d-af28-e5e6ee949c96","Type":"ContainerStarted","Data":"d244d473db3d7a967d966ef5ee128bb9581d4af97df1a92174b50d2ef2d23ae8"} Jul 22 04:34:33.778042 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778001 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778057 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh8vx\" (UniqueName: \"kubernetes.io/projected/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-kube-api-access-hh8vx\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778273 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778165 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778273 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778207 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778273 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778254 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-metrics-client-ca\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778285 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.778411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.778342 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-grpc-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.796502 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.795776 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fw8ht" podStartSLOduration=2.81599707 podStartE2EDuration="3.795758115s" totalImagesPullingTime="914.718876ms" totalInitContainerRuntime="65.042169ms" isStatefulPod=true podCreationTimestamp="2026-07-22 04:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:33.79387135 +0000 UTC m=+88.842139609" watchObservedRunningTime="2026-07-22 04:34:33.795758115 +0000 UTC m=+88.844026375" Jul 22 04:34:33.879655 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879625 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-grpc-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.879776 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879734 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.879843 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879773 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh8vx\" (UniqueName: \"kubernetes.io/projected/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-kube-api-access-hh8vx\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.879843 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.879951 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879851 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.879951 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.879889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.880079 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.880008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-metrics-client-ca\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.880079 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.880058 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.881081 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.881051 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-metrics-client-ca\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.883389 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.882532 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.883389 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.882594 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-grpc-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.883389 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.882956 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.883389 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.883332 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-tls\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.883642 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.883625 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.884312 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.884284 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:33.887861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:33.887835 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh8vx\" (UniqueName: \"kubernetes.io/projected/7dad0cf6-a79a-44a1-97e4-f7c0213b66dd-kube-api-access-hh8vx\") pod \"thanos-querier-5cf98d6887-f8bfj\" (UID: \"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd\") " pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:34.016737 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.016707 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:34.148614 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.148524 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5cf98d6887-f8bfj"] Jul 22 04:34:34.151137 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:34.151105 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dad0cf6_a79a_44a1_97e4_f7c0213b66dd.slice/crio-4e1de6baa7f0b56777d41b16c1d6f591e06c28b33b41c52b3bba8b1eb22179f8 WatchSource:0}: Error finding container 4e1de6baa7f0b56777d41b16c1d6f591e06c28b33b41c52b3bba8b1eb22179f8: Status 404 returned error can't find the container with id 4e1de6baa7f0b56777d41b16c1d6f591e06c28b33b41c52b3bba8b1eb22179f8 Jul 22 04:34:34.781227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.781163 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" event={"ID":"ad1b7570-747d-45bd-aaf8-ebc12c73c426","Type":"ContainerStarted","Data":"436e150e0a8d21af0b12b6807a3d5a23bca7fc75b3083f44195d8a1c8f2fe266"} Jul 22 04:34:34.781643 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.781232 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" event={"ID":"ad1b7570-747d-45bd-aaf8-ebc12c73c426","Type":"ContainerStarted","Data":"53420535d184c117b7ed4133dd6cf7daf71224066f1d7cc595438cbb88187a34"} Jul 22 04:34:34.782713 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.782683 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"4e1de6baa7f0b56777d41b16c1d6f591e06c28b33b41c52b3bba8b1eb22179f8"} Jul 22 04:34:34.804466 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:34.804427 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-vv7b8" podStartSLOduration=3.061453374 podStartE2EDuration="4.804413343s" totalImagesPullingTime="1.742959969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:34.803693183 +0000 UTC m=+89.851961438" watchObservedRunningTime="2026-07-22 04:34:34.804413343 +0000 UTC m=+89.852681602" Jul 22 04:34:35.123550 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.123480 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:35.488158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.488093 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj"] Jul 22 04:34:35.491152 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.491128 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.493326 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.493306 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 04:34:35.493457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.493364 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5fkz5\"" Jul 22 04:34:35.508236 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.505231 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj"] Jul 22 04:34:35.593948 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.593917 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwv5t\" (UniqueName: \"kubernetes.io/projected/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-kube-api-access-gwv5t\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.594076 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.593974 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.694859 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.694825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwv5t\" (UniqueName: \"kubernetes.io/projected/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-kube-api-access-gwv5t\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.695010 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.694885 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.697446 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.697418 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.705472 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.705445 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwv5t\" (UniqueName: \"kubernetes.io/projected/290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2-kube-api-access-gwv5t\") pod \"monitoring-plugin-7b7b7bc8bf-hk2tj\" (UID: \"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.799937 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.799910 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:35.914270 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.914235 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-696fb57c6b-lkmgb"] Jul 22 04:34:35.917644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.917617 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.920198 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920113 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 04:34:35.920328 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920265 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 04:34:35.920399 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920388 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 04:34:35.920494 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920449 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hqsxc\"" Jul 22 04:34:35.920617 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920590 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 04:34:35.920734 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.920669 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 04:34:35.926224 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.926139 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 04:34:35.928321 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.928234 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-696fb57c6b-lkmgb"] Jul 22 04:34:35.998107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998065 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998121 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998171 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-federate-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998212 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998268 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-metrics-client-ca\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998336 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-serving-certs-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:35.998495 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:35.998362 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmx95\" (UniqueName: \"kubernetes.io/projected/0a62ecf7-3880-458b-a3ef-66543d66bd0f-kube-api-access-xmx95\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099126 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099056 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099126 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099095 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099128 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-federate-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099158 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099177 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-metrics-client-ca\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099537 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099394 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-serving-certs-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.099537 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.099436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmx95\" (UniqueName: \"kubernetes.io/projected/0a62ecf7-3880-458b-a3ef-66543d66bd0f-kube-api-access-xmx95\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.100370 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.100096 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-metrics-client-ca\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.100370 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.100143 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.100370 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.100342 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0a62ecf7-3880-458b-a3ef-66543d66bd0f-serving-certs-ca-bundle\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.101761 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.101733 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.101869 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.101779 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.101980 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.101960 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-federate-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.102012 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.101960 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0a62ecf7-3880-458b-a3ef-66543d66bd0f-telemeter-client-tls\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.112061 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.112018 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmx95\" (UniqueName: \"kubernetes.io/projected/0a62ecf7-3880-458b-a3ef-66543d66bd0f-kube-api-access-xmx95\") pod \"telemeter-client-696fb57c6b-lkmgb\" (UID: \"0a62ecf7-3880-458b-a3ef-66543d66bd0f\") " pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.231346 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.230883 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" Jul 22 04:34:36.237519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.237241 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj"] Jul 22 04:34:36.372834 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.372802 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-696fb57c6b-lkmgb"] Jul 22 04:34:36.376967 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:36.376946 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a62ecf7_3880_458b_a3ef_66543d66bd0f.slice/crio-abeb1b8e25367c7b3e227c0a8e8eb8353af0ae61a13823c45764a1d0b84f55e5 WatchSource:0}: Error finding container abeb1b8e25367c7b3e227c0a8e8eb8353af0ae61a13823c45764a1d0b84f55e5: Status 404 returned error can't find the container with id abeb1b8e25367c7b3e227c0a8e8eb8353af0ae61a13823c45764a1d0b84f55e5 Jul 22 04:34:36.793181 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.793137 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" event={"ID":"0a62ecf7-3880-458b-a3ef-66543d66bd0f","Type":"ContainerStarted","Data":"abeb1b8e25367c7b3e227c0a8e8eb8353af0ae61a13823c45764a1d0b84f55e5"} Jul 22 04:34:36.798469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.794513 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" event={"ID":"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2","Type":"ContainerStarted","Data":"be1a665c26ac41fe2090ec25129868c51759081aabecd0a1d00650c883d80bf1"} Jul 22 04:34:36.798469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.796569 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"4d8638e653e3c4a76241bd394cb7ec48b1afc493052567a62c28bc2d5c3ed236"} Jul 22 04:34:36.798469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.796631 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"e5fe52c78c34fe83ccc85a98e4cbd2da1d32f52c34e8f0bacc9b2bd0265e5f15"} Jul 22 04:34:36.798469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:36.796669 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"328b1ed54f36e7890a458f5c8041a7b22e76649f0143de3540cfd5677955b3be"} Jul 22 04:34:37.805040 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:37.804982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"49a512a7a21948f87fc044a231ca70f4c208b415547be7dddceaa4dca55ed0e4"} Jul 22 04:34:37.805516 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:37.805126 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"1c4cf4a85864ea9ba1a6155d64ef60b1ef5229d9147832e03cfcbd940173734f"} Jul 22 04:34:37.805516 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:37.805185 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" event={"ID":"7dad0cf6-a79a-44a1-97e4-f7c0213b66dd","Type":"ContainerStarted","Data":"82a9d0ee27ac67ea92cd655e2bd12d8047a183cf8e3e601935d5b2151a2d6083"} Jul 22 04:34:37.805516 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:37.805232 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:37.830695 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:37.829672 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" podStartSLOduration=1.908815527 podStartE2EDuration="4.829651202s" totalImagesPullingTime="2.920835675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:37.827183627 +0000 UTC m=+92.875451878" watchObservedRunningTime="2026-07-22 04:34:37.829651202 +0000 UTC m=+92.877919464" Jul 22 04:34:38.810521 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:38.810486 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" event={"ID":"0a62ecf7-3880-458b-a3ef-66543d66bd0f","Type":"ContainerStarted","Data":"3768e2ea0239d750862e1bf6bb28543a66caecfbe4b1a73e7fce094f9269a7d8"} Jul 22 04:34:38.811995 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:38.811965 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" event={"ID":"290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2","Type":"ContainerStarted","Data":"5856f6086e3ffb1eb316b39534b7b4ff62140bc3c636bc4ef8ca50f6d5322363"} Jul 22 04:34:38.812124 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:38.812016 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:38.816738 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:38.816711 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" Jul 22 04:34:38.831419 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:38.828463 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-hk2tj" podStartSLOduration=2.177216854 podStartE2EDuration="3.828443306s" totalImagesPullingTime="1.651226452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:38.827677884 +0000 UTC m=+93.875946145" watchObservedRunningTime="2026-07-22 04:34:38.828443306 +0000 UTC m=+93.876711566" Jul 22 04:34:39.817818 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:39.817778 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" event={"ID":"0a62ecf7-3880-458b-a3ef-66543d66bd0f","Type":"ContainerStarted","Data":"5ab4b0077e88573583c0bd7d00870673da05cc28c008727e6009ee4fbc6d4343"} Jul 22 04:34:39.818293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:39.817851 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" event={"ID":"0a62ecf7-3880-458b-a3ef-66543d66bd0f","Type":"ContainerStarted","Data":"f9ef46e42ef731472bf8c3a8028fb79dc4287af8020fab5cbccc58c14a388049"} Jul 22 04:34:40.137927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.137834 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" podUID="f145647b-e904-4020-a03f-533c7710b1e8" containerName="registry" containerID="cri-o://17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d" gracePeriod=30 Jul 22 04:34:40.375404 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.375381 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:40.394260 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.394161 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-696fb57c6b-lkmgb" podStartSLOduration=2.340658579 podStartE2EDuration="5.394138555s" totalImagesPullingTime="3.053479976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:39.839198776 +0000 UTC m=+94.887467058" watchObservedRunningTime="2026-07-22 04:34:40.394138555 +0000 UTC m=+95.442406816" Jul 22 04:34:40.538676 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538625 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538717 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538742 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538761 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538787 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538818 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56ndq\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.538872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.538847 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.539418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.539353 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token\") pod \"f145647b-e904-4020-a03f-533c7710b1e8\" (UID: \"f145647b-e904-4020-a03f-533c7710b1e8\") " Jul 22 04:34:40.539418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.539370 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:34:40.539418 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.539376 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:34:40.539647 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.539605 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-trusted-ca\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.539647 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.539626 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f145647b-e904-4020-a03f-533c7710b1e8-registry-certificates\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.541275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.541252 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:34:40.541522 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.541485 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:34:40.541619 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.541570 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:34:40.541619 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.541613 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:34:40.541697 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.541668 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq" (OuterVolumeSpecName: "kube-api-access-56ndq") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "kube-api-access-56ndq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:34:40.547781 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.547755 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f145647b-e904-4020-a03f-533c7710b1e8" (UID: "f145647b-e904-4020-a03f-533c7710b1e8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:34:40.640164 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640127 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f145647b-e904-4020-a03f-533c7710b1e8-ca-trust-extracted\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.640164 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640158 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-installation-pull-secrets\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.640164 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640168 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f145647b-e904-4020-a03f-533c7710b1e8-image-registry-private-configuration\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.640164 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640177 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-registry-tls\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.640409 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640185 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56ndq\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-kube-api-access-56ndq\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.640409 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.640193 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f145647b-e904-4020-a03f-533c7710b1e8-bound-sa-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:34:40.710292 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.710226 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dd6957979-g82wp" Jul 22 04:34:40.823409 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.823374 2555 generic.go:356] "Generic (PLEG): container finished" podID="f145647b-e904-4020-a03f-533c7710b1e8" containerID="17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d" exitCode=0 Jul 22 04:34:40.823841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.823422 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" Jul 22 04:34:40.823841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.823463 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" event={"ID":"f145647b-e904-4020-a03f-533c7710b1e8","Type":"ContainerDied","Data":"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d"} Jul 22 04:34:40.823841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.823508 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6674bbbc96-lgzxz" event={"ID":"f145647b-e904-4020-a03f-533c7710b1e8","Type":"ContainerDied","Data":"d6c3ca758f1fdf95b801bef2a814535d6c81a374795cd4745eb001c04af31a8f"} Jul 22 04:34:40.823841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.823533 2555 scope.go:117] "RemoveContainer" containerID="17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d" Jul 22 04:34:40.834471 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.834449 2555 scope.go:117] "RemoveContainer" containerID="17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d" Jul 22 04:34:40.834821 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:34:40.834794 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d\": container with ID starting with 17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d not found: ID does not exist" containerID="17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d" Jul 22 04:34:40.834944 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.834826 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d"} err="failed to get container status \"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d\": rpc error: code = NotFound desc = could not find container \"17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d\": container with ID starting with 17e715020e1e95170a4d74ce6a9e56829d8b39181ddc1fd72c59d5921b9c091d not found: ID does not exist" Jul 22 04:34:40.845405 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.845372 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:34:40.848972 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:40.848946 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6674bbbc96-lgzxz"] Jul 22 04:34:41.432524 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:41.430576 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f145647b-e904-4020-a03f-533c7710b1e8" path="/var/lib/kubelet/pods/f145647b-e904-4020-a03f-533c7710b1e8/volumes" Jul 22 04:34:43.818763 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:43.818735 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5cf98d6887-f8bfj" Jul 22 04:34:43.837168 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:43.837138 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" event={"ID":"82f74c97-a149-42b8-a578-a979c50fb5cf","Type":"ContainerStarted","Data":"67e33abf68aa5d8b0e274bfddc0a2aba55376259fd5a8d0f6daf7f3f4887bb7c"} Jul 22 04:34:43.853406 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:43.853346 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-648b548986-d45zs" podStartSLOduration=4.7823665890000004 podStartE2EDuration="28.853325252s" totalImagesPullingTime="24.070958663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:43.852924845 +0000 UTC m=+98.901193102" watchObservedRunningTime="2026-07-22 04:34:43.853325252 +0000 UTC m=+98.901593513" Jul 22 04:34:45.993969 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.993930 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-sxgrg"] Jul 22 04:34:45.994426 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.994253 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f145647b-e904-4020-a03f-533c7710b1e8" containerName="registry" Jul 22 04:34:45.994426 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.994265 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f145647b-e904-4020-a03f-533c7710b1e8" containerName="registry" Jul 22 04:34:45.994426 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.994317 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f145647b-e904-4020-a03f-533c7710b1e8" containerName="registry" Jul 22 04:34:45.997402 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.997384 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:34:45.999637 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.999615 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 04:34:45.999755 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.999640 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vgzqd\"" Jul 22 04:34:45.999755 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:45.999641 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 04:34:46.006658 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.006634 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-sxgrg"] Jul 22 04:34:46.185603 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.185576 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjrcb\" (UniqueName: \"kubernetes.io/projected/c46c07cc-63c3-4276-8f63-6e142f164fa7-kube-api-access-tjrcb\") pod \"downloads-5bc67dcd46-sxgrg\" (UID: \"c46c07cc-63c3-4276-8f63-6e142f164fa7\") " pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:34:46.286124 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.286099 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjrcb\" (UniqueName: \"kubernetes.io/projected/c46c07cc-63c3-4276-8f63-6e142f164fa7-kube-api-access-tjrcb\") pod \"downloads-5bc67dcd46-sxgrg\" (UID: \"c46c07cc-63c3-4276-8f63-6e142f164fa7\") " pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:34:46.293912 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.293886 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjrcb\" (UniqueName: \"kubernetes.io/projected/c46c07cc-63c3-4276-8f63-6e142f164fa7-kube-api-access-tjrcb\") pod \"downloads-5bc67dcd46-sxgrg\" (UID: \"c46c07cc-63c3-4276-8f63-6e142f164fa7\") " pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:34:46.306726 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.306683 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:34:46.441960 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.441933 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-sxgrg"] Jul 22 04:34:46.444403 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:34:46.444376 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46c07cc_63c3_4276_8f63_6e142f164fa7.slice/crio-3d4d3dae6ca60c7975f8adccb95cbb9be2b747abbcc60099531a6ff71bbb7556 WatchSource:0}: Error finding container 3d4d3dae6ca60c7975f8adccb95cbb9be2b747abbcc60099531a6ff71bbb7556: Status 404 returned error can't find the container with id 3d4d3dae6ca60c7975f8adccb95cbb9be2b747abbcc60099531a6ff71bbb7556 Jul 22 04:34:46.850057 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:46.850003 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-sxgrg" event={"ID":"c46c07cc-63c3-4276-8f63-6e142f164fa7","Type":"ContainerStarted","Data":"3d4d3dae6ca60c7975f8adccb95cbb9be2b747abbcc60099531a6ff71bbb7556"} Jul 22 04:34:50.709216 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:50.709183 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m2gjh" Jul 22 04:34:57.630239 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.630191 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:34:57.744822 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.744784 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:34:57.745000 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.744919 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.747940 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.747917 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 04:34:57.747940 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.747934 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vn4zg\"" Jul 22 04:34:57.748174 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.748141 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 04:34:57.748236 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.748209 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 04:34:57.748483 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.748380 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 04:34:57.748605 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.748493 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 04:34:57.788010 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.787980 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.788172 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.788035 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.788172 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.788103 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.788284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.788174 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.788284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.788196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.788284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.788217 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z42wv\" (UniqueName: \"kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889209 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889209 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889209 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889461 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889277 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889461 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889320 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889461 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889343 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z42wv\" (UniqueName: \"kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.889980 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.889956 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.890187 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.890105 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.890282 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.890261 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.892266 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.892241 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.892356 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.892273 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:57.901829 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:57.901790 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z42wv\" (UniqueName: \"kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv\") pod \"console-76f5f47574-qmq2p\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:34:58.057474 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:34:58.057436 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:02.452043 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.451989 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:35:02.454308 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:35:02.454258 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd8a409_bd2a_4bc2_abb9_fbaccdcaa2ca.slice/crio-9e5fc5c5f60e2b6a6e202c5c4368e54a91ec20da111d9359462ad90b0fe9d1cb WatchSource:0}: Error finding container 9e5fc5c5f60e2b6a6e202c5c4368e54a91ec20da111d9359462ad90b0fe9d1cb: Status 404 returned error can't find the container with id 9e5fc5c5f60e2b6a6e202c5c4368e54a91ec20da111d9359462ad90b0fe9d1cb Jul 22 04:35:02.922103 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.922059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5f47574-qmq2p" event={"ID":"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca","Type":"ContainerStarted","Data":"9e5fc5c5f60e2b6a6e202c5c4368e54a91ec20da111d9359462ad90b0fe9d1cb"} Jul 22 04:35:02.923989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.923941 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-sxgrg" event={"ID":"c46c07cc-63c3-4276-8f63-6e142f164fa7","Type":"ContainerStarted","Data":"9d8bfc9ab7bf6cbdff1d550e4f3ca95b1f7ea914b0ea0c5d32ff1e2cfbaef8b1"} Jul 22 04:35:02.924193 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.924169 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:35:02.942736 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.942690 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-sxgrg" podStartSLOduration=1.998173212 podStartE2EDuration="17.942674282s" totalImagesPullingTime="15.94450107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:35:02.941198584 +0000 UTC m=+117.989466847" watchObservedRunningTime="2026-07-22 04:35:02.942674282 +0000 UTC m=+117.990942542" Jul 22 04:35:02.943525 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:02.943499 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-sxgrg" Jul 22 04:35:06.949510 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:06.949417 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5f47574-qmq2p" event={"ID":"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca","Type":"ContainerStarted","Data":"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28"} Jul 22 04:35:06.983565 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:06.983490 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76f5f47574-qmq2p" podStartSLOduration=5.887873422 podStartE2EDuration="9.983465427s" totalImagesPullingTime="4.095592005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:35:06.98298472 +0000 UTC m=+122.031252979" watchObservedRunningTime="2026-07-22 04:35:06.983465427 +0000 UTC m=+122.031733687" Jul 22 04:35:08.058576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:08.058537 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:08.059048 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:08.058653 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:08.064199 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:08.064172 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:08.962500 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:08.962466 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:17.659622 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:17.659580 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:35:42.680576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:42.680537 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76f5f47574-qmq2p" podUID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" containerName="console" containerID="cri-o://c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28" gracePeriod=15 Jul 22 04:35:42.945173 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:42.945119 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:43.048056 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048015 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048211 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048076 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048211 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048098 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048312 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048204 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048312 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048251 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048312 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048301 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z42wv\" (UniqueName: \"kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv\") pod \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\" (UID: \"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca\") " Jul 22 04:35:43.048582 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048555 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca" (OuterVolumeSpecName: "service-ca") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:35:43.048725 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048604 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:35:43.048725 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.048662 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config" (OuterVolumeSpecName: "console-config") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:35:43.050457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.050429 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv" (OuterVolumeSpecName: "kube-api-access-z42wv") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "kube-api-access-z42wv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:35:43.050571 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.050504 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:35:43.050571 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.050520 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" (UID: "acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:35:43.086822 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.086798 2555 generic.go:356] "Generic (PLEG): container finished" podID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" containerID="c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28" exitCode=0 Jul 22 04:35:43.086938 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.086856 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5f47574-qmq2p" Jul 22 04:35:43.086938 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.086886 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5f47574-qmq2p" event={"ID":"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca","Type":"ContainerDied","Data":"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28"} Jul 22 04:35:43.087093 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.086959 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5f47574-qmq2p" event={"ID":"acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca","Type":"ContainerDied","Data":"9e5fc5c5f60e2b6a6e202c5c4368e54a91ec20da111d9359462ad90b0fe9d1cb"} Jul 22 04:35:43.087093 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.086981 2555 scope.go:117] "RemoveContainer" containerID="c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28" Jul 22 04:35:43.096328 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.096312 2555 scope.go:117] "RemoveContainer" containerID="c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28" Jul 22 04:35:43.096582 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:35:43.096558 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28\": container with ID starting with c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28 not found: ID does not exist" containerID="c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28" Jul 22 04:35:43.096620 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.096590 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28"} err="failed to get container status \"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28\": rpc error: code = NotFound desc = could not find container \"c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28\": container with ID starting with c2b22d31e95c84da033dc6d0cb6b846ab72f80347deb764abae662b7556a9b28 not found: ID does not exist" Jul 22 04:35:43.106284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.106265 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:35:43.109351 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.109332 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76f5f47574-qmq2p"] Jul 22 04:35:43.148960 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.148940 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-service-ca\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.149078 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.148964 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-oauth-serving-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.149078 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.148981 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-serving-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.149078 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.148993 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-config\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.149078 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.149052 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-console-oauth-config\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.149078 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.149065 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z42wv\" (UniqueName: \"kubernetes.io/projected/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca-kube-api-access-z42wv\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:35:43.429202 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:35:43.429173 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" path="/var/lib/kubelet/pods/acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca/volumes" Jul 22 04:37:14.533380 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.533338 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt"] Jul 22 04:37:14.533854 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.533659 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" containerName="console" Jul 22 04:37:14.533854 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.533671 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" containerName="console" Jul 22 04:37:14.533854 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.533719 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd8a409-bd2a-4bc2-abb9-fbaccdcaa2ca" containerName="console" Jul 22 04:37:14.535530 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.535511 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.537783 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.537763 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 04:37:14.538793 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.538769 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 04:37:14.538892 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.538769 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-t4lvw\"" Jul 22 04:37:14.545606 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.545582 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt"] Jul 22 04:37:14.552406 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.552380 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ctpz\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-kube-api-access-2ctpz\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.552522 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.552414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.652950 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.652907 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ctpz\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-kube-api-access-2ctpz\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.652950 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.652954 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.661453 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.661421 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.661659 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.661634 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ctpz\" (UniqueName: \"kubernetes.io/projected/3483ce50-dace-4c95-82ad-28f15da477a3-kube-api-access-2ctpz\") pod \"cert-manager-cainjector-559cdd8c87-h4qgt\" (UID: \"3483ce50-dace-4c95-82ad-28f15da477a3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.845016 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.844927 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" Jul 22 04:37:14.978410 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:14.978376 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt"] Jul 22 04:37:14.981549 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:37:14.981517 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3483ce50_dace_4c95_82ad_28f15da477a3.slice/crio-c83de2c43bcabb40ce6f094045e6d722e045c926f47be6b79f06cf6737489a07 WatchSource:0}: Error finding container c83de2c43bcabb40ce6f094045e6d722e045c926f47be6b79f06cf6737489a07: Status 404 returned error can't find the container with id c83de2c43bcabb40ce6f094045e6d722e045c926f47be6b79f06cf6737489a07 Jul 22 04:37:15.415639 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:15.415605 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" event={"ID":"3483ce50-dace-4c95-82ad-28f15da477a3","Type":"ContainerStarted","Data":"c83de2c43bcabb40ce6f094045e6d722e045c926f47be6b79f06cf6737489a07"} Jul 22 04:37:18.430067 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:18.430000 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" event={"ID":"3483ce50-dace-4c95-82ad-28f15da477a3","Type":"ContainerStarted","Data":"67ba3029e9c9e6206ee78d5bf373c701c6de35acb28eba99cfd5d75a94603c90"} Jul 22 04:37:18.447829 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:18.447780 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h4qgt" podStartSLOduration=1.5593526450000001 podStartE2EDuration="4.44776418s" totalImagesPullingTime="2.888411535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:18.446244428 +0000 UTC m=+253.494512688" watchObservedRunningTime="2026-07-22 04:37:18.44776418 +0000 UTC m=+253.496032438" Jul 22 04:37:25.429593 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.429561 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4"] Jul 22 04:37:25.431663 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.431646 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.434037 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.434008 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:37:25.434161 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.434044 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-qlcm6\"" Jul 22 04:37:25.435100 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.435083 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 04:37:25.441296 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.441274 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4"] Jul 22 04:37:25.541121 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.541074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ssvn\" (UniqueName: \"kubernetes.io/projected/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-kube-api-access-5ssvn\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.541296 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.541142 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.641563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.641520 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.641750 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.641604 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ssvn\" (UniqueName: \"kubernetes.io/projected/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-kube-api-access-5ssvn\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.641938 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.641917 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.654293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.654261 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ssvn\" (UniqueName: \"kubernetes.io/projected/f10ad3f0-2113-457a-ba79-ee7d3f30b45e-kube-api-access-5ssvn\") pod \"openshift-lws-operator-fd8ccff4c-9dcf4\" (UID: \"f10ad3f0-2113-457a-ba79-ee7d3f30b45e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.741813 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.741726 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" Jul 22 04:37:25.880881 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:25.880845 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4"] Jul 22 04:37:25.884290 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:37:25.884254 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf10ad3f0_2113_457a_ba79_ee7d3f30b45e.slice/crio-790f8afab8aa2f3c99c853e7963ae2c5b5a3908916250789a21b1448d67b1f35 WatchSource:0}: Error finding container 790f8afab8aa2f3c99c853e7963ae2c5b5a3908916250789a21b1448d67b1f35: Status 404 returned error can't find the container with id 790f8afab8aa2f3c99c853e7963ae2c5b5a3908916250789a21b1448d67b1f35 Jul 22 04:37:26.460921 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:26.460874 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" event={"ID":"f10ad3f0-2113-457a-ba79-ee7d3f30b45e","Type":"ContainerStarted","Data":"790f8afab8aa2f3c99c853e7963ae2c5b5a3908916250789a21b1448d67b1f35"} Jul 22 04:37:28.469746 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:28.469653 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" event={"ID":"f10ad3f0-2113-457a-ba79-ee7d3f30b45e","Type":"ContainerStarted","Data":"9d478860f7750031eef51a7c24be00cf0d132d139b17b98f3d32aacb639cec2b"} Jul 22 04:37:28.486776 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:28.486717 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9dcf4" podStartSLOduration=1.215837177 podStartE2EDuration="3.486700232s" totalImagesPullingTime="2.270863055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:28.483944049 +0000 UTC m=+263.532212308" watchObservedRunningTime="2026-07-22 04:37:28.486700232 +0000 UTC m=+263.534968490" Jul 22 04:37:47.932623 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.932581 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd"] Jul 22 04:37:47.935178 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.935153 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:47.938252 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.938218 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 04:37:47.938404 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.938307 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nzjg5\"" Jul 22 04:37:47.938404 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.938380 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 04:37:47.938695 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.938675 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 04:37:47.938898 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.938883 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 04:37:47.961887 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:47.961857 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd"] Jul 22 04:37:48.027609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.027575 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j2ws\" (UniqueName: \"kubernetes.io/projected/31944654-99d5-4b4b-9b2a-fe3b7db42e29-kube-api-access-7j2ws\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.027609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.027616 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-manifests\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.027853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.027636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.027853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.027762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-webhook-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.027853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.027801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-tmp\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.128853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.128810 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-webhook-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.128853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.128852 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-tmp\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.129113 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.128906 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j2ws\" (UniqueName: \"kubernetes.io/projected/31944654-99d5-4b4b-9b2a-fe3b7db42e29-kube-api-access-7j2ws\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.129113 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.128929 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-manifests\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.129113 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.128947 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.129448 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.129428 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-manifests\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.131298 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.131277 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/31944654-99d5-4b4b-9b2a-fe3b7db42e29-tmp\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.131453 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.131435 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.131500 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.131477 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/31944654-99d5-4b4b-9b2a-fe3b7db42e29-webhook-cert\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.138828 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.138802 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j2ws\" (UniqueName: \"kubernetes.io/projected/31944654-99d5-4b4b-9b2a-fe3b7db42e29-kube-api-access-7j2ws\") pod \"opendatahub-operator-controller-manager-5567574457-s8tcd\" (UID: \"31944654-99d5-4b4b-9b2a-fe3b7db42e29\") " pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.245179 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.245084 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:48.403805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.403761 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd"] Jul 22 04:37:48.406871 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:37:48.406844 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31944654_99d5_4b4b_9b2a_fe3b7db42e29.slice/crio-ebff4f3fc1b5ccb2c3baf40a331b863f615d3a9dd2d8fc147c7192b3b998b215 WatchSource:0}: Error finding container ebff4f3fc1b5ccb2c3baf40a331b863f615d3a9dd2d8fc147c7192b3b998b215: Status 404 returned error can't find the container with id ebff4f3fc1b5ccb2c3baf40a331b863f615d3a9dd2d8fc147c7192b3b998b215 Jul 22 04:37:48.548853 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:48.548818 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" event={"ID":"31944654-99d5-4b4b-9b2a-fe3b7db42e29","Type":"ContainerStarted","Data":"ebff4f3fc1b5ccb2c3baf40a331b863f615d3a9dd2d8fc147c7192b3b998b215"} Jul 22 04:37:52.568255 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:52.568213 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" event={"ID":"31944654-99d5-4b4b-9b2a-fe3b7db42e29","Type":"ContainerStarted","Data":"cff15ffc422598c98b4b9383d4a84bcbbb94d10ceae4073fde3d2539eb8d4bb1"} Jul 22 04:37:53.574046 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:53.573992 2555 generic.go:356] "Generic (PLEG): container finished" podID="31944654-99d5-4b4b-9b2a-fe3b7db42e29" containerID="cff15ffc422598c98b4b9383d4a84bcbbb94d10ceae4073fde3d2539eb8d4bb1" exitCode=0 Jul 22 04:37:53.574629 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:53.574059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" event={"ID":"31944654-99d5-4b4b-9b2a-fe3b7db42e29","Type":"ContainerDied","Data":"cff15ffc422598c98b4b9383d4a84bcbbb94d10ceae4073fde3d2539eb8d4bb1"} Jul 22 04:37:54.579696 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:54.579662 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" event={"ID":"31944654-99d5-4b4b-9b2a-fe3b7db42e29","Type":"ContainerStarted","Data":"111221ca7bd922b7834a8f1cdf94132aadb4b2bd9b3aac87d58b41536b294740"} Jul 22 04:37:54.580123 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:54.579777 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:37:54.601856 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:37:54.601800 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" podStartSLOduration=2.926383255 podStartE2EDuration="7.601783246s" totalImagesPullingTime="3.642890523s" totalInitContainerRuntime="1.032509468s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:54.60022378 +0000 UTC m=+289.648492053" watchObservedRunningTime="2026-07-22 04:37:54.601783246 +0000 UTC m=+289.650051505" Jul 22 04:38:05.314287 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:05.314247 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:38:05.319828 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:05.319802 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:38:05.322704 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:05.322680 2555 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 04:38:05.586546 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:05.586460 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5567574457-s8tcd" Jul 22 04:38:07.844157 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:07.844116 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dqv4n"] Jul 22 04:38:07.847414 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:07.847391 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:07.850042 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:07.850002 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 04:38:07.850214 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:07.850197 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-w9jrb\"" Jul 22 04:38:07.856107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:07.856079 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dqv4n"] Jul 22 04:38:08.002052 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.001988 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b89822bc-24c2-4c21-b858-0d592985075f-cert\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.002234 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.002145 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksc4d\" (UniqueName: \"kubernetes.io/projected/b89822bc-24c2-4c21-b858-0d592985075f-kube-api-access-ksc4d\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.103642 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.103530 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b89822bc-24c2-4c21-b858-0d592985075f-cert\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.103642 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.103605 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksc4d\" (UniqueName: \"kubernetes.io/projected/b89822bc-24c2-4c21-b858-0d592985075f-kube-api-access-ksc4d\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.106121 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.106096 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b89822bc-24c2-4c21-b858-0d592985075f-cert\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.111598 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.111574 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksc4d\" (UniqueName: \"kubernetes.io/projected/b89822bc-24c2-4c21-b858-0d592985075f-kube-api-access-ksc4d\") pod \"odh-model-controller-7998c67dc4-dqv4n\" (UID: \"b89822bc-24c2-4c21-b858-0d592985075f\") " pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.158958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.158918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:08.291432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.291285 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-dqv4n"] Jul 22 04:38:08.294275 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:08.294248 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89822bc_24c2_4c21_b858_0d592985075f.slice/crio-3c020a57bb9876d44794b4971b68de0d265d4e41f1629c2f21da3f396b74f2c4 WatchSource:0}: Error finding container 3c020a57bb9876d44794b4971b68de0d265d4e41f1629c2f21da3f396b74f2c4: Status 404 returned error can't find the container with id 3c020a57bb9876d44794b4971b68de0d265d4e41f1629c2f21da3f396b74f2c4 Jul 22 04:38:08.296152 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.296134 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:38:08.488596 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.488510 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r7cm5"] Jul 22 04:38:08.492966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.492947 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.495480 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.495457 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 04:38:08.505719 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.505691 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r7cm5"] Jul 22 04:38:08.607483 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.607445 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjz77\" (UniqueName: \"kubernetes.io/projected/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-kube-api-access-qjz77\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.607680 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.607512 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-tls-certs\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.639246 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.639215 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" event={"ID":"b89822bc-24c2-4c21-b858-0d592985075f","Type":"ContainerStarted","Data":"3c020a57bb9876d44794b4971b68de0d265d4e41f1629c2f21da3f396b74f2c4"} Jul 22 04:38:08.708664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.708629 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjz77\" (UniqueName: \"kubernetes.io/projected/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-kube-api-access-qjz77\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.708834 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.708708 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-tls-certs\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.711220 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.711197 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-tls-certs\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.716511 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.716490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjz77\" (UniqueName: \"kubernetes.io/projected/2bc3a940-3aa9-429c-b1c1-ad00677d8f53-kube-api-access-qjz77\") pod \"model-serving-api-54d89bd5fc-r7cm5\" (UID: \"2bc3a940-3aa9-429c-b1c1-ad00677d8f53\") " pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.803435 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.803396 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:08.941155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:08.941118 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r7cm5"] Jul 22 04:38:08.944125 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:08.944096 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc3a940_3aa9_429c_b1c1_ad00677d8f53.slice/crio-7fe6cc8e5ca0dc302f5bd107d48779d11cebab8b1033928ec40023f7cfc4d0b6 WatchSource:0}: Error finding container 7fe6cc8e5ca0dc302f5bd107d48779d11cebab8b1033928ec40023f7cfc4d0b6: Status 404 returned error can't find the container with id 7fe6cc8e5ca0dc302f5bd107d48779d11cebab8b1033928ec40023f7cfc4d0b6 Jul 22 04:38:09.387372 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.387333 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln"] Jul 22 04:38:09.391249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.391206 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.393980 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.393725 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 04:38:09.393980 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.393786 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 04:38:09.395402 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.395374 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8jt7v\"" Jul 22 04:38:09.395558 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.395538 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 04:38:09.395660 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.395639 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 04:38:09.399381 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.399360 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln"] Jul 22 04:38:09.517109 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.517017 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7476468-64fe-4598-ba2b-e5775a458b0d-tmp\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.517315 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.517154 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pcgm\" (UniqueName: \"kubernetes.io/projected/a7476468-64fe-4598-ba2b-e5775a458b0d-kube-api-access-9pcgm\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.517315 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.517210 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7476468-64fe-4598-ba2b-e5775a458b0d-tls-certs\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.618198 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.618156 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7476468-64fe-4598-ba2b-e5775a458b0d-tmp\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.618398 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.618236 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pcgm\" (UniqueName: \"kubernetes.io/projected/a7476468-64fe-4598-ba2b-e5775a458b0d-kube-api-access-9pcgm\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.618398 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.618277 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7476468-64fe-4598-ba2b-e5775a458b0d-tls-certs\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.621197 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.621168 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7476468-64fe-4598-ba2b-e5775a458b0d-tmp\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.621739 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.621717 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a7476468-64fe-4598-ba2b-e5775a458b0d-tls-certs\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.626842 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.626812 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pcgm\" (UniqueName: \"kubernetes.io/projected/a7476468-64fe-4598-ba2b-e5775a458b0d-kube-api-access-9pcgm\") pod \"kube-auth-proxy-d644bdf8d-qgsln\" (UID: \"a7476468-64fe-4598-ba2b-e5775a458b0d\") " pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.647476 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.647392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" event={"ID":"2bc3a940-3aa9-429c-b1c1-ad00677d8f53","Type":"ContainerStarted","Data":"7fe6cc8e5ca0dc302f5bd107d48779d11cebab8b1033928ec40023f7cfc4d0b6"} Jul 22 04:38:09.705640 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.705593 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" Jul 22 04:38:09.905620 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:09.904643 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln"] Jul 22 04:38:10.350774 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.350737 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:10.357911 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.357879 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.360458 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.360425 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 04:38:10.360610 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.360478 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 04:38:10.360610 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.360556 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-25qrg\"" Jul 22 04:38:10.360610 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.360596 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 04:38:10.360770 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.360677 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 04:38:10.370433 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.370406 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:10.429723 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.429930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429739 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2pvp\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.429930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.429930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429842 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.429930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429890 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.430158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.429979 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.430158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.430038 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530801 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530766 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530828 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530883 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530914 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2pvp\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530953 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.530989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.530980 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.531357 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.531013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.531982 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.531940 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.533734 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.533706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.533841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.533706 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.533902 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.533879 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.533987 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.533962 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.539365 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.539342 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.539507 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.539485 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2pvp\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp\") pod \"istiod-openshift-gateway-7574459656-pfjb8\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:10.655474 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.655374 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" event={"ID":"a7476468-64fe-4598-ba2b-e5775a458b0d","Type":"ContainerStarted","Data":"e98dbcd1061ae35febf81281d1bf64fce294a8f6abbb66658ae9304590658b73"} Jul 22 04:38:10.671607 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:10.671572 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:12.664641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:12.664611 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:12.669860 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:12.669818 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23e4f54_e58d_443c_9365_fbd744e640ff.slice/crio-f759d32232d9087a66e9b5e421954f41b3aefb5c5f113cee76ea46b4c25d2466 WatchSource:0}: Error finding container f759d32232d9087a66e9b5e421954f41b3aefb5c5f113cee76ea46b4c25d2466: Status 404 returned error can't find the container with id f759d32232d9087a66e9b5e421954f41b3aefb5c5f113cee76ea46b4c25d2466 Jul 22 04:38:12.674409 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:12.674381 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:12.693192 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:12.693127 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" podStartSLOduration=1.4657637000000001 podStartE2EDuration="5.693092638s" totalImagesPullingTime="4.227328938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:12.690260705 +0000 UTC m=+307.738528956" watchObservedRunningTime="2026-07-22 04:38:12.693092638 +0000 UTC m=+307.741360899" Jul 22 04:38:13.680254 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.680220 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" event={"ID":"d23e4f54-e58d-443c-9365-fbd744e640ff","Type":"ContainerStarted","Data":"f759d32232d9087a66e9b5e421954f41b3aefb5c5f113cee76ea46b4c25d2466"} Jul 22 04:38:13.681567 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.681542 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" event={"ID":"2bc3a940-3aa9-429c-b1c1-ad00677d8f53","Type":"ContainerStarted","Data":"1563c6acb83b9c52312124f1ebd0fa40443552ad358436cb0f680a6af69a9ec9"} Jul 22 04:38:13.681681 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.681621 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:13.682959 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.682940 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" event={"ID":"b89822bc-24c2-4c21-b858-0d592985075f","Type":"ContainerStarted","Data":"afae9d2598758b0b729e6a65c821154eb0d6a61e4fdc4bf614bd7b6206e800d6"} Jul 22 04:38:13.684328 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.684309 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" event={"ID":"a7476468-64fe-4598-ba2b-e5775a458b0d","Type":"ContainerStarted","Data":"122e4acf95036b2409dfb6d4862d795dd318d70f1746292b090a057b04a6f3eb"} Jul 22 04:38:13.698749 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.698697 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" podStartSLOduration=2.117731102 podStartE2EDuration="5.698681028s" totalImagesPullingTime="3.580949926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:13.697066534 +0000 UTC m=+308.745334790" watchObservedRunningTime="2026-07-22 04:38:13.698681028 +0000 UTC m=+308.746949284" Jul 22 04:38:13.712412 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:13.712356 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-d644bdf8d-qgsln" podStartSLOduration=2.049727661 podStartE2EDuration="4.712340503s" totalImagesPullingTime="2.662612842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:13.711079977 +0000 UTC m=+308.759348237" watchObservedRunningTime="2026-07-22 04:38:13.712340503 +0000 UTC m=+308.760608761" Jul 22 04:38:15.496107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:15.496056 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:15.496442 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:15.496135 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:15.695493 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:15.695458 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" event={"ID":"d23e4f54-e58d-443c-9365-fbd744e640ff","Type":"ContainerStarted","Data":"78f87dc94235369b12a886eae12433805cc46a0af7812af7a95afa8b01583f31"} Jul 22 04:38:15.695677 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:15.695597 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:15.721098 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:15.721045 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" podStartSLOduration=2.897786269 podStartE2EDuration="5.721005706s" totalImagesPullingTime="2.823219437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:15.719803737 +0000 UTC m=+310.768071997" watchObservedRunningTime="2026-07-22 04:38:15.721005706 +0000 UTC m=+310.769273965" Jul 22 04:38:16.703107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.703067 2555 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-pfjb8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:38:16.703568 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.703126 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:38:16.920410 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.920369 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:16.925410 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.925382 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.929315 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.929285 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c69h9\"" Jul 22 04:38:16.951167 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.951136 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:16.991836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991733 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.991836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991775 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.991836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991801 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991868 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991950 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.991983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slbsr\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.992009 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992104 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.992084 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:16.992280 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:16.992124 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.051899 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.051863 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj"] Jul 22 04:38:17.055326 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.055302 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.071683 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.071652 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj"] Jul 22 04:38:17.092967 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.092935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.092977 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03259dd4-d21f-454a-85c5-b2ea70adfa79-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093013 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093096 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093126 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093163 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093212 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2dfp\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-kube-api-access-h2dfp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093246 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093312 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093374 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093368 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093411 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093481 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093492 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093512 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093549 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slbsr\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093583 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.093673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.093626 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.094753 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.094624 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.095073 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.095048 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.095191 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.095087 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.095420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.095397 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.095887 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.095858 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.096291 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.096263 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.108385 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.108349 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slbsr\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.108563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.108541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-nj8hz\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.194608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194568 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03259dd4-d21f-454a-85c5-b2ea70adfa79-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.194608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.194870 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.194870 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194832 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.194870 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194869 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2dfp\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-kube-api-access-h2dfp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195064 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195064 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194957 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195064 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.194983 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195232 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195285 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195233 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195408 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195384 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195492 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195428 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195492 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195443 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.195492 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.195475 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03259dd4-d21f-454a-85c5-b2ea70adfa79-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.197332 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.197311 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.197568 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.197547 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.202882 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.202849 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2dfp\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-kube-api-access-h2dfp\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.203045 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.202958 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03259dd4-d21f-454a-85c5-b2ea70adfa79-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678j5fmj\" (UID: \"03259dd4-d21f-454a-85c5-b2ea70adfa79\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.236057 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.236004 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:17.365823 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.365793 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:17.419908 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.419384 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:17.521110 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.521081 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj"] Jul 22 04:38:17.522956 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:17.522925 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03259dd4_d21f_454a_85c5_b2ea70adfa79.slice/crio-6ffcc1e84c63e34d83c79ad783af99cd28e94c05c2a89461b742fc308ae0647c WatchSource:0}: Error finding container 6ffcc1e84c63e34d83c79ad783af99cd28e94c05c2a89461b742fc308ae0647c: Status 404 returned error can't find the container with id 6ffcc1e84c63e34d83c79ad783af99cd28e94c05c2a89461b742fc308ae0647c Jul 22 04:38:17.709713 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.709677 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" event={"ID":"03259dd4-d21f-454a-85c5-b2ea70adfa79","Type":"ContainerStarted","Data":"6ffcc1e84c63e34d83c79ad783af99cd28e94c05c2a89461b742fc308ae0647c"} Jul 22 04:38:17.711000 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:17.710974 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" event={"ID":"403814c1-0b35-450b-98d1-a4ca9468380a","Type":"ContainerStarted","Data":"bc250fce34678881d03d779407a563c68ecde6d046f4e995281cda304fd27954"} Jul 22 04:38:18.100998 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.100953 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxltv"] Jul 22 04:38:18.105141 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.105110 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.106718 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.106683 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5"] Jul 22 04:38:18.107910 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.107864 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sxbxw\"" Jul 22 04:38:18.108421 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.108161 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 22 04:38:18.110630 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.110609 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.116205 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.113274 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 22 04:38:18.116205 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.113593 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6lkqt\"" Jul 22 04:38:18.116205 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.115984 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxltv"] Jul 22 04:38:18.126149 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.125779 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5"] Jul 22 04:38:18.206168 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.206122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thknw\" (UniqueName: \"kubernetes.io/projected/241e0eae-d237-46f1-9151-c9302d843c75-kube-api-access-thknw\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.206351 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.206215 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mnz5\" (UniqueName: \"kubernetes.io/projected/b81bea95-d97a-4049-9f97-c79f21dbf0f3-kube-api-access-6mnz5\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.206470 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.206442 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.206553 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.206493 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/241e0eae-d237-46f1-9151-c9302d843c75-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.307161 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.307117 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.307368 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.307172 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/241e0eae-d237-46f1-9151-c9302d843c75-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.307368 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.307203 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thknw\" (UniqueName: \"kubernetes.io/projected/241e0eae-d237-46f1-9151-c9302d843c75-kube-api-access-thknw\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.307368 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.307270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mnz5\" (UniqueName: \"kubernetes.io/projected/b81bea95-d97a-4049-9f97-c79f21dbf0f3-kube-api-access-6mnz5\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.307368 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:38:18.307293 2555 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 22 04:38:18.307592 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:38:18.307372 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert podName:b81bea95-d97a-4049-9f97-c79f21dbf0f3 nodeName:}" failed. No retries permitted until 2026-07-22 04:38:18.807348799 +0000 UTC m=+313.855617047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert") pod "llmisvc-controller-manager-777c6bd7d5-4bfl5" (UID: "b81bea95-d97a-4049-9f97-c79f21dbf0f3") : secret "llmisvc-webhook-server-cert" not found Jul 22 04:38:18.310639 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.310607 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/241e0eae-d237-46f1-9151-c9302d843c75-cert\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.327443 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.327387 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mnz5\" (UniqueName: \"kubernetes.io/projected/b81bea95-d97a-4049-9f97-c79f21dbf0f3-kube-api-access-6mnz5\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.330928 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.330894 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thknw\" (UniqueName: \"kubernetes.io/projected/241e0eae-d237-46f1-9151-c9302d843c75-kube-api-access-thknw\") pod \"kserve-controller-manager-d6f4b65b8-wxltv\" (UID: \"241e0eae-d237-46f1-9151-c9302d843c75\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.440437 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.440341 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:18.615484 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.615317 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wxltv"] Jul 22 04:38:18.617196 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:18.617162 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241e0eae_d237_46f1_9151_c9302d843c75.slice/crio-61b8049027f5321e5cfcb89bcb6a7ce2741915ffba20221c75b9d105cc3601be WatchSource:0}: Error finding container 61b8049027f5321e5cfcb89bcb6a7ce2741915ffba20221c75b9d105cc3601be: Status 404 returned error can't find the container with id 61b8049027f5321e5cfcb89bcb6a7ce2741915ffba20221c75b9d105cc3601be Jul 22 04:38:18.718831 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.718748 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" event={"ID":"241e0eae-d237-46f1-9151-c9302d843c75","Type":"ContainerStarted","Data":"61b8049027f5321e5cfcb89bcb6a7ce2741915ffba20221c75b9d105cc3601be"} Jul 22 04:38:18.817063 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.812950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:18.819623 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:18.819558 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b81bea95-d97a-4049-9f97-c79f21dbf0f3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4bfl5\" (UID: \"b81bea95-d97a-4049-9f97-c79f21dbf0f3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:19.048995 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:19.048947 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:19.233753 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:19.233705 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5"] Jul 22 04:38:19.549753 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:19.549693 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:19.550067 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:19.549940 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerName="discovery" containerID="cri-o://78f87dc94235369b12a886eae12433805cc46a0af7812af7a95afa8b01583f31" gracePeriod=30 Jul 22 04:38:19.550909 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:19.550669 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:20.733471 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:20.733384 2555 generic.go:356] "Generic (PLEG): container finished" podID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerID="78f87dc94235369b12a886eae12433805cc46a0af7812af7a95afa8b01583f31" exitCode=0 Jul 22 04:38:20.733930 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:20.733479 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" event={"ID":"d23e4f54-e58d-443c-9365-fbd744e640ff","Type":"ContainerDied","Data":"78f87dc94235369b12a886eae12433805cc46a0af7812af7a95afa8b01583f31"} Jul 22 04:38:20.913666 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:38:20.913623 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb81bea95_d97a_4049_9f97_c79f21dbf0f3.slice/crio-89cf610f91f99ec5c1daba1016050af18b1b50e0b55604a972a11c01b75e6d49 WatchSource:0}: Error finding container 89cf610f91f99ec5c1daba1016050af18b1b50e0b55604a972a11c01b75e6d49: Status 404 returned error can't find the container with id 89cf610f91f99ec5c1daba1016050af18b1b50e0b55604a972a11c01b75e6d49 Jul 22 04:38:20.969776 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:20.969749 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:21.001408 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.001372 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.001503 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.001445 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.001503 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.001477 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.006921 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.006893 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.006985 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.006958 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.007043 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.006987 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:38:21.038214 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038189 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038240 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038278 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038455 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038433 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038511 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038482 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038560 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038511 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038614 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038554 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2pvp\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp\") pod \"d23e4f54-e58d-443c-9365-fbd744e640ff\" (UID: \"d23e4f54-e58d-443c-9365-fbd744e640ff\") " Jul 22 04:38:21.038664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038616 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:38:21.038889 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.038869 2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-ca-configmap\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.040960 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.040856 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:38:21.040960 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.040952 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp" (OuterVolumeSpecName: "kube-api-access-p2pvp") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "kube-api-access-p2pvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:38:21.041192 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.041172 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs" (OuterVolumeSpecName: "local-certs") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:21.041256 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.041188 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:38:21.041256 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.041169 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:38:21.041319 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.041297 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts" (OuterVolumeSpecName: "cacerts") pod "d23e4f54-e58d-443c-9365-fbd744e640ff" (UID: "d23e4f54-e58d-443c-9365-fbd744e640ff"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:38:21.139781 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.139738 2555 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d23e4f54-e58d-443c-9365-fbd744e640ff-local-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.140081 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.140063 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.140195 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.140183 2555 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-kubeconfig\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.140400 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.140336 2555 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-istio-csr-dns-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.140513 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.140499 2555 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d23e4f54-e58d-443c-9365-fbd744e640ff-cacerts\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.140595 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.140584 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2pvp\" (UniqueName: \"kubernetes.io/projected/d23e4f54-e58d-443c-9365-fbd744e640ff-kube-api-access-p2pvp\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:21.740386 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.740303 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" Jul 22 04:38:21.740829 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.740302 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8" event={"ID":"d23e4f54-e58d-443c-9365-fbd744e640ff","Type":"ContainerDied","Data":"f759d32232d9087a66e9b5e421954f41b3aefb5c5f113cee76ea46b4c25d2466"} Jul 22 04:38:21.740829 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.740452 2555 scope.go:117] "RemoveContainer" containerID="78f87dc94235369b12a886eae12433805cc46a0af7812af7a95afa8b01583f31" Jul 22 04:38:21.742276 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.742237 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" event={"ID":"03259dd4-d21f-454a-85c5-b2ea70adfa79","Type":"ContainerStarted","Data":"30c28688c6dd259780c5f66284ddfe629a59e9276a573dad87db148320662ab8"} Jul 22 04:38:21.744203 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.744161 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" event={"ID":"241e0eae-d237-46f1-9151-c9302d843c75","Type":"ContainerStarted","Data":"7f2d936948921dbc4e54fd0d2eb8c319546505184591078e0ffdfb5ac98f73f0"} Jul 22 04:38:21.744321 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.744245 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:21.745576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.745546 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" event={"ID":"b81bea95-d97a-4049-9f97-c79f21dbf0f3","Type":"ContainerStarted","Data":"89cf610f91f99ec5c1daba1016050af18b1b50e0b55604a972a11c01b75e6d49"} Jul 22 04:38:21.747290 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.747270 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" event={"ID":"403814c1-0b35-450b-98d1-a4ca9468380a","Type":"ContainerStarted","Data":"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd"} Jul 22 04:38:21.776315 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.776264 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" podStartSLOduration=1.300379006 podStartE2EDuration="4.776246416s" totalImagesPullingTime="3.47586741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:21.774516736 +0000 UTC m=+316.822784996" watchObservedRunningTime="2026-07-22 04:38:21.776246416 +0000 UTC m=+316.824514674" Jul 22 04:38:21.792944 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.792892 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" podStartSLOduration=1.411490986 podStartE2EDuration="3.792873569s" totalImagesPullingTime="2.381382583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:21.791851823 +0000 UTC m=+316.840120083" watchObservedRunningTime="2026-07-22 04:38:21.792873569 +0000 UTC m=+316.841141865" Jul 22 04:38:21.806964 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.806928 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:21.813250 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.813220 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-pfjb8"] Jul 22 04:38:21.830881 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:21.830818 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" podStartSLOduration=2.244452492 podStartE2EDuration="5.830793605s" totalImagesPullingTime="3.586341113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:21.828970528 +0000 UTC m=+316.877238788" watchObservedRunningTime="2026-07-22 04:38:21.830793605 +0000 UTC m=+316.879061864" Jul 22 04:38:22.236465 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.236421 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:22.238432 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.238395 2555 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-nj8hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 04:38:22.238596 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.238463 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 04:38:22.366794 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.366746 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:22.372217 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.372188 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:22.754658 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.754590 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:22.755653 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.755551 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678j5fmj" Jul 22 04:38:22.808212 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:22.808175 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:23.236664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.236571 2555 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-nj8hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 04:38:23.236664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.236647 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 04:38:23.440187 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.440150 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" path="/var/lib/kubelet/pods/d23e4f54-e58d-443c-9365-fbd744e640ff/volumes" Jul 22 04:38:23.760441 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.760406 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" event={"ID":"b81bea95-d97a-4049-9f97-c79f21dbf0f3","Type":"ContainerStarted","Data":"a70638efbabfd173876534433ddebbf0ac534235f430379377d6f0b921d1d85b"} Jul 22 04:38:23.760913 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.760719 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:38:23.777896 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:23.777844 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" podStartSLOduration=3.022340831 podStartE2EDuration="5.777829761s" totalImagesPullingTime="2.75548893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:23.774830385 +0000 UTC m=+318.823098637" watchObservedRunningTime="2026-07-22 04:38:23.777829761 +0000 UTC m=+318.826098019" Jul 22 04:38:24.236449 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:24.236405 2555 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-nj8hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" start-of-body= Jul 22 04:38:24.236640 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:24.236472 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.27:15021/healthz/ready\": dial tcp 10.132.0.27:15021: connect: connection refused" Jul 22 04:38:24.692386 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:24.692353 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-dqv4n" Jul 22 04:38:24.694173 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:24.694153 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-r7cm5" Jul 22 04:38:24.765360 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:24.765319 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" containerID="cri-o://5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd" gracePeriod=30 Jul 22 04:38:30.022085 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.022049 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:30.132269 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132159 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132269 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132225 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132269 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132258 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132285 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132349 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132395 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132445 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132483 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slbsr\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr\") pod \"403814c1-0b35-450b-98d1-a4ca9468380a\" (UID: \"403814c1-0b35-450b-98d1-a4ca9468380a\") " Jul 22 04:38:30.132611 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132530 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:30.132968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132644 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data" (OuterVolumeSpecName: "istio-data") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:30.132968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132805 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.132968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132826 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.132968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.132877 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:38:30.133137 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.133075 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:30.133291 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.133263 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:30.134712 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.134690 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:38:30.134854 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.134835 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:38:30.134977 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.134954 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr" (OuterVolumeSpecName: "kube-api-access-slbsr") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "kube-api-access-slbsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:38:30.135049 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.134993 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token" (OuterVolumeSpecName: "istio-token") pod "403814c1-0b35-450b-98d1-a4ca9468380a" (UID: "403814c1-0b35-450b-98d1-a4ca9468380a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:38:30.234307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234267 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234298 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234307 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234319 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/403814c1-0b35-450b-98d1-a4ca9468380a-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234655 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234330 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/403814c1-0b35-450b-98d1-a4ca9468380a-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234655 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234340 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/403814c1-0b35-450b-98d1-a4ca9468380a-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.234655 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.234350 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slbsr\" (UniqueName: \"kubernetes.io/projected/403814c1-0b35-450b-98d1-a4ca9468380a-kube-api-access-slbsr\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:38:30.793762 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.793729 2555 generic.go:356] "Generic (PLEG): container finished" podID="403814c1-0b35-450b-98d1-a4ca9468380a" containerID="5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd" exitCode=0 Jul 22 04:38:30.793997 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.793778 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" event={"ID":"403814c1-0b35-450b-98d1-a4ca9468380a","Type":"ContainerDied","Data":"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd"} Jul 22 04:38:30.793997 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.793781 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" Jul 22 04:38:30.793997 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.793818 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz" event={"ID":"403814c1-0b35-450b-98d1-a4ca9468380a","Type":"ContainerDied","Data":"bc250fce34678881d03d779407a563c68ecde6d046f4e995281cda304fd27954"} Jul 22 04:38:30.793997 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.793840 2555 scope.go:117] "RemoveContainer" containerID="5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd" Jul 22 04:38:30.802689 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.802666 2555 scope.go:117] "RemoveContainer" containerID="5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd" Jul 22 04:38:30.802985 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:38:30.802962 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd\": container with ID starting with 5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd not found: ID does not exist" containerID="5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd" Jul 22 04:38:30.803057 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.802996 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd"} err="failed to get container status \"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd\": rpc error: code = NotFound desc = could not find container \"5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd\": container with ID starting with 5f264b56f26b5fc7d53e0b17183c8973f72f6d6cf2a0b670ba2008df371a05bd not found: ID does not exist" Jul 22 04:38:30.815841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.815816 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:30.819359 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:30.819330 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-nj8hz"] Jul 22 04:38:31.430951 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:31.430913 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" path="/var/lib/kubelet/pods/403814c1-0b35-450b-98d1-a4ca9468380a/volumes" Jul 22 04:38:52.759876 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:52.759845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wxltv" Jul 22 04:38:54.767414 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:38:54.767379 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4bfl5" Jul 22 04:39:08.783810 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.783774 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:08.784211 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784189 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" Jul 22 04:39:08.784211 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784207 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" Jul 22 04:39:08.784284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784224 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerName="discovery" Jul 22 04:39:08.784284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784230 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerName="discovery" Jul 22 04:39:08.784348 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784289 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="403814c1-0b35-450b-98d1-a4ca9468380a" containerName="istio-proxy" Jul 22 04:39:08.784348 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.784297 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d23e4f54-e58d-443c-9365-fbd744e640ff" containerName="discovery" Jul 22 04:39:08.788910 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.788887 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:08.791644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.791614 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 04:39:08.792660 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.792643 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 04:39:08.792743 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.792667 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-h67w4\"" Jul 22 04:39:08.801115 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.801093 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:08.842070 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.842042 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xndfp\" (UniqueName: \"kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp\") pod \"kuadrant-operator-catalog-rk7sw\" (UID: \"11bdc213-f434-41cd-9722-a38aa2c04e99\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:08.942864 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.942837 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xndfp\" (UniqueName: \"kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp\") pod \"kuadrant-operator-catalog-rk7sw\" (UID: \"11bdc213-f434-41cd-9722-a38aa2c04e99\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:08.950266 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:08.950236 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xndfp\" (UniqueName: \"kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp\") pod \"kuadrant-operator-catalog-rk7sw\" (UID: \"11bdc213-f434-41cd-9722-a38aa2c04e99\") " pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:09.101035 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.100958 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:09.153068 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.152544 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:09.226968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.226941 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:09.229728 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:09.229702 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bdc213_f434_41cd_9722_a38aa2c04e99.slice/crio-e58b094442dc05f62ccba9a8877e7b627926bdc680151d5e34a21271caa78a0c WatchSource:0}: Error finding container e58b094442dc05f62ccba9a8877e7b627926bdc680151d5e34a21271caa78a0c: Status 404 returned error can't find the container with id e58b094442dc05f62ccba9a8877e7b627926bdc680151d5e34a21271caa78a0c Jul 22 04:39:09.361270 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.361215 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r55h8"] Jul 22 04:39:09.365668 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.365650 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:09.372316 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.372294 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r55h8"] Jul 22 04:39:09.446549 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.446524 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jh49\" (UniqueName: \"kubernetes.io/projected/6aa3220a-7609-422d-90f9-3e94369fde6e-kube-api-access-9jh49\") pod \"kuadrant-operator-catalog-r55h8\" (UID: \"6aa3220a-7609-422d-90f9-3e94369fde6e\") " pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:09.546954 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.546926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jh49\" (UniqueName: \"kubernetes.io/projected/6aa3220a-7609-422d-90f9-3e94369fde6e-kube-api-access-9jh49\") pod \"kuadrant-operator-catalog-r55h8\" (UID: \"6aa3220a-7609-422d-90f9-3e94369fde6e\") " pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:09.554796 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.554769 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jh49\" (UniqueName: \"kubernetes.io/projected/6aa3220a-7609-422d-90f9-3e94369fde6e-kube-api-access-9jh49\") pod \"kuadrant-operator-catalog-r55h8\" (UID: \"6aa3220a-7609-422d-90f9-3e94369fde6e\") " pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:09.676150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.676094 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:09.807331 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.807307 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r55h8"] Jul 22 04:39:09.809867 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:09.809827 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa3220a_7609_422d_90f9_3e94369fde6e.slice/crio-5ff854e93c5b594e27ac2f58887792e047df78b82bbafe223da437fb80442ace WatchSource:0}: Error finding container 5ff854e93c5b594e27ac2f58887792e047df78b82bbafe223da437fb80442ace: Status 404 returned error can't find the container with id 5ff854e93c5b594e27ac2f58887792e047df78b82bbafe223da437fb80442ace Jul 22 04:39:09.976245 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.976169 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" event={"ID":"6aa3220a-7609-422d-90f9-3e94369fde6e","Type":"ContainerStarted","Data":"5ff854e93c5b594e27ac2f58887792e047df78b82bbafe223da437fb80442ace"} Jul 22 04:39:09.977353 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:09.977327 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" event={"ID":"11bdc213-f434-41cd-9722-a38aa2c04e99","Type":"ContainerStarted","Data":"e58b094442dc05f62ccba9a8877e7b627926bdc680151d5e34a21271caa78a0c"} Jul 22 04:39:11.989524 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:11.989480 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" event={"ID":"6aa3220a-7609-422d-90f9-3e94369fde6e","Type":"ContainerStarted","Data":"266bd2cb0c11cbbe91c5118d0dd0551b7815c73958c2ed49bd74b3e1be33e204"} Jul 22 04:39:11.991407 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:11.991365 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" event={"ID":"11bdc213-f434-41cd-9722-a38aa2c04e99","Type":"ContainerStarted","Data":"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3"} Jul 22 04:39:11.991563 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:11.991451 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" podUID="11bdc213-f434-41cd-9722-a38aa2c04e99" containerName="registry-server" containerID="cri-o://0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3" gracePeriod=2 Jul 22 04:39:12.004547 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.004487 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" podStartSLOduration=1.63038966 podStartE2EDuration="3.004464722s" totalImagesPullingTime="1.374075062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:12.00293421 +0000 UTC m=+367.051202469" watchObservedRunningTime="2026-07-22 04:39:12.004464722 +0000 UTC m=+367.052732982" Jul 22 04:39:12.017178 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.017135 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" podStartSLOduration=2.063346211 podStartE2EDuration="4.017094652s" totalImagesPullingTime="1.953748441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:12.016344768 +0000 UTC m=+367.064613028" watchObservedRunningTime="2026-07-22 04:39:12.017094652 +0000 UTC m=+367.065362911" Jul 22 04:39:12.234626 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.234602 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:12.270030 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.270003 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xndfp\" (UniqueName: \"kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp\") pod \"11bdc213-f434-41cd-9722-a38aa2c04e99\" (UID: \"11bdc213-f434-41cd-9722-a38aa2c04e99\") " Jul 22 04:39:12.271915 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.271894 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp" (OuterVolumeSpecName: "kube-api-access-xndfp") pod "11bdc213-f434-41cd-9722-a38aa2c04e99" (UID: "11bdc213-f434-41cd-9722-a38aa2c04e99"). InnerVolumeSpecName "kube-api-access-xndfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:39:12.371444 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.371421 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xndfp\" (UniqueName: \"kubernetes.io/projected/11bdc213-f434-41cd-9722-a38aa2c04e99-kube-api-access-xndfp\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:39:12.998155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.998108 2555 generic.go:356] "Generic (PLEG): container finished" podID="11bdc213-f434-41cd-9722-a38aa2c04e99" containerID="0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3" exitCode=0 Jul 22 04:39:12.998592 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.998169 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" Jul 22 04:39:12.998592 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.998209 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" event={"ID":"11bdc213-f434-41cd-9722-a38aa2c04e99","Type":"ContainerDied","Data":"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3"} Jul 22 04:39:12.998592 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.998266 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-rk7sw" event={"ID":"11bdc213-f434-41cd-9722-a38aa2c04e99","Type":"ContainerDied","Data":"e58b094442dc05f62ccba9a8877e7b627926bdc680151d5e34a21271caa78a0c"} Jul 22 04:39:12.998592 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:12.998289 2555 scope.go:117] "RemoveContainer" containerID="0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3" Jul 22 04:39:13.007917 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:13.007892 2555 scope.go:117] "RemoveContainer" containerID="0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3" Jul 22 04:39:13.008213 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:39:13.008193 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3\": container with ID starting with 0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3 not found: ID does not exist" containerID="0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3" Jul 22 04:39:13.008275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:13.008223 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3"} err="failed to get container status \"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3\": rpc error: code = NotFound desc = could not find container \"0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3\": container with ID starting with 0efe90cac9687385b0ddf267fa7f91e4e8a74ad7f1d3f7c378a13114eaf414b3 not found: ID does not exist" Jul 22 04:39:13.018689 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:13.018653 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:13.022266 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:13.022234 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-rk7sw"] Jul 22 04:39:13.430559 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:13.430483 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11bdc213-f434-41cd-9722-a38aa2c04e99" path="/var/lib/kubelet/pods/11bdc213-f434-41cd-9722-a38aa2c04e99/volumes" Jul 22 04:39:19.676947 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:19.676899 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:19.677467 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:19.676969 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:19.700286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:19.700260 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:20.055698 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:20.055670 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-r55h8" Jul 22 04:39:40.878143 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.878104 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6"] Jul 22 04:39:40.878615 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.878446 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11bdc213-f434-41cd-9722-a38aa2c04e99" containerName="registry-server" Jul 22 04:39:40.878615 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.878457 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="11bdc213-f434-41cd-9722-a38aa2c04e99" containerName="registry-server" Jul 22 04:39:40.878615 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.878531 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="11bdc213-f434-41cd-9722-a38aa2c04e99" containerName="registry-server" Jul 22 04:39:40.881598 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.881578 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:40.885528 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.885509 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-l4r56\"" Jul 22 04:39:40.893893 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.893873 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6"] Jul 22 04:39:40.986447 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:40.986422 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kltqd\" (UniqueName: \"kubernetes.io/projected/8eff9a5e-1131-4be2-bb35-067dfda38a46-kube-api-access-kltqd\") pod \"authorino-operator-75cc8d5cb7-z2sl6\" (UID: \"8eff9a5e-1131-4be2-bb35-067dfda38a46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:41.087681 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:41.087658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kltqd\" (UniqueName: \"kubernetes.io/projected/8eff9a5e-1131-4be2-bb35-067dfda38a46-kube-api-access-kltqd\") pod \"authorino-operator-75cc8d5cb7-z2sl6\" (UID: \"8eff9a5e-1131-4be2-bb35-067dfda38a46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:41.100044 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:41.099996 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kltqd\" (UniqueName: \"kubernetes.io/projected/8eff9a5e-1131-4be2-bb35-067dfda38a46-kube-api-access-kltqd\") pod \"authorino-operator-75cc8d5cb7-z2sl6\" (UID: \"8eff9a5e-1131-4be2-bb35-067dfda38a46\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:41.191785 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:41.191714 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:41.317366 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:41.317340 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6"] Jul 22 04:39:41.320051 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:41.320009 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eff9a5e_1131_4be2_bb35_067dfda38a46.slice/crio-809f18f03aebec2df18b65ebb5e9e1bd6b09a29aefba1acbfc7c4942d3851bc3 WatchSource:0}: Error finding container 809f18f03aebec2df18b65ebb5e9e1bd6b09a29aefba1acbfc7c4942d3851bc3: Status 404 returned error can't find the container with id 809f18f03aebec2df18b65ebb5e9e1bd6b09a29aefba1acbfc7c4942d3851bc3 Jul 22 04:39:42.137394 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:42.137313 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" event={"ID":"8eff9a5e-1131-4be2-bb35-067dfda38a46","Type":"ContainerStarted","Data":"809f18f03aebec2df18b65ebb5e9e1bd6b09a29aefba1acbfc7c4942d3851bc3"} Jul 22 04:39:44.147956 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.147917 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" event={"ID":"8eff9a5e-1131-4be2-bb35-067dfda38a46","Type":"ContainerStarted","Data":"6247a92ed6bbfe069425891eb5019a549002bc7925a54841b06efde4ed3a2df1"} Jul 22 04:39:44.148391 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.148051 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:44.152861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.152822 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn"] Jul 22 04:39:44.156411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.156387 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:44.158983 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.158963 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 22 04:39:44.159105 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.159006 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-f8bw4\"" Jul 22 04:39:44.165180 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.165153 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn"] Jul 22 04:39:44.170085 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.169995 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" podStartSLOduration=2.029842987 podStartE2EDuration="4.169980085s" totalImagesPullingTime="2.140137098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:44.169203517 +0000 UTC m=+399.217471779" watchObservedRunningTime="2026-07-22 04:39:44.169980085 +0000 UTC m=+399.218248345" Jul 22 04:39:44.317415 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.317380 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgkb2\" (UniqueName: \"kubernetes.io/projected/ca4c5a77-c948-45f0-ad44-da876ea54fbd-kube-api-access-wgkb2\") pod \"dns-operator-controller-manager-864dd78c4c-lmdvn\" (UID: \"ca4c5a77-c948-45f0-ad44-da876ea54fbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:44.418257 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.418195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgkb2\" (UniqueName: \"kubernetes.io/projected/ca4c5a77-c948-45f0-ad44-da876ea54fbd-kube-api-access-wgkb2\") pod \"dns-operator-controller-manager-864dd78c4c-lmdvn\" (UID: \"ca4c5a77-c948-45f0-ad44-da876ea54fbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:44.433189 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.433160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgkb2\" (UniqueName: \"kubernetes.io/projected/ca4c5a77-c948-45f0-ad44-da876ea54fbd-kube-api-access-wgkb2\") pod \"dns-operator-controller-manager-864dd78c4c-lmdvn\" (UID: \"ca4c5a77-c948-45f0-ad44-da876ea54fbd\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:44.467945 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.467918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:44.602649 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:44.602620 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn"] Jul 22 04:39:44.604675 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:44.604635 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4c5a77_c948_45f0_ad44_da876ea54fbd.slice/crio-2056a15b2dd457638567e5ca77503e5525d0a04a8bb2132e0bdcbb7f48e94581 WatchSource:0}: Error finding container 2056a15b2dd457638567e5ca77503e5525d0a04a8bb2132e0bdcbb7f48e94581: Status 404 returned error can't find the container with id 2056a15b2dd457638567e5ca77503e5525d0a04a8bb2132e0bdcbb7f48e94581 Jul 22 04:39:45.154431 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:45.154395 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" event={"ID":"ca4c5a77-c948-45f0-ad44-da876ea54fbd","Type":"ContainerStarted","Data":"2056a15b2dd457638567e5ca77503e5525d0a04a8bb2132e0bdcbb7f48e94581"} Jul 22 04:39:47.166519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:47.166474 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" event={"ID":"ca4c5a77-c948-45f0-ad44-da876ea54fbd","Type":"ContainerStarted","Data":"b94f0d1f761b09bc542b62d6f79de30a2ec7e6c03839630f8b73a8e0ee79b284"} Jul 22 04:39:47.166983 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:47.166545 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:39:47.187490 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:47.187438 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" podStartSLOduration=1.419859341 podStartE2EDuration="3.18741959s" totalImagesPullingTime="1.767560249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:47.185896325 +0000 UTC m=+402.234164584" watchObservedRunningTime="2026-07-22 04:39:47.18741959 +0000 UTC m=+402.235687938" Jul 22 04:39:48.848254 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.848219 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz"] Jul 22 04:39:48.851727 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.851701 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:48.855015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.854969 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v4dp9\"" Jul 22 04:39:48.864748 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.864725 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz"] Jul 22 04:39:48.955588 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.955566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:48.955697 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:48.955623 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vww49\" (UniqueName: \"kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.056176 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.056144 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vww49\" (UniqueName: \"kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.056276 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.056209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.056572 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.056553 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.064250 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.064217 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vww49\" (UniqueName: \"kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.162189 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.162129 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:49.297442 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:49.297410 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz"] Jul 22 04:39:49.300636 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:49.300601 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94931e26_a6e3_4d4f_a40b_3c07c0b31354.slice/crio-42923658d902d05aa96e16c9941cce110b2e0d30e61d25b966bdd2621b0bd0cf WatchSource:0}: Error finding container 42923658d902d05aa96e16c9941cce110b2e0d30e61d25b966bdd2621b0bd0cf: Status 404 returned error can't find the container with id 42923658d902d05aa96e16c9941cce110b2e0d30e61d25b966bdd2621b0bd0cf Jul 22 04:39:50.186032 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.185980 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" event={"ID":"94931e26-a6e3-4d4f-a40b-3c07c0b31354","Type":"ContainerStarted","Data":"42923658d902d05aa96e16c9941cce110b2e0d30e61d25b966bdd2621b0bd0cf"} Jul 22 04:39:50.860843 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.860805 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml"] Jul 22 04:39:50.864507 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.864486 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:50.867035 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.866998 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-v5kp6\"" Jul 22 04:39:50.876455 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.876393 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml"] Jul 22 04:39:50.972606 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:50.972576 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqsqv\" (UniqueName: \"kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv\") pod \"limitador-operator-controller-manager-859d7b78bd-g4cml\" (UID: \"317808c1-178a-41db-9710-ebb584788cf3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:51.073861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:51.073837 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqsqv\" (UniqueName: \"kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv\") pod \"limitador-operator-controller-manager-859d7b78bd-g4cml\" (UID: \"317808c1-178a-41db-9710-ebb584788cf3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:51.088912 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:51.088887 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqsqv\" (UniqueName: \"kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv\") pod \"limitador-operator-controller-manager-859d7b78bd-g4cml\" (UID: \"317808c1-178a-41db-9710-ebb584788cf3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:51.179297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:51.179244 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:51.340509 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:51.340485 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml"] Jul 22 04:39:51.342853 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:39:51.342815 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317808c1_178a_41db_9710_ebb584788cf3.slice/crio-4b3d2bab996f5490e1daec2abf11b9d920f57896016c5a91b8231fe61f6784af WatchSource:0}: Error finding container 4b3d2bab996f5490e1daec2abf11b9d920f57896016c5a91b8231fe61f6784af: Status 404 returned error can't find the container with id 4b3d2bab996f5490e1daec2abf11b9d920f57896016c5a91b8231fe61f6784af Jul 22 04:39:52.199762 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:52.199724 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" event={"ID":"317808c1-178a-41db-9710-ebb584788cf3","Type":"ContainerStarted","Data":"4b3d2bab996f5490e1daec2abf11b9d920f57896016c5a91b8231fe61f6784af"} Jul 22 04:39:54.211886 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.211851 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" event={"ID":"94931e26-a6e3-4d4f-a40b-3c07c0b31354","Type":"ContainerStarted","Data":"b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512"} Jul 22 04:39:54.212938 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.212746 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:39:54.214410 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.214382 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" event={"ID":"317808c1-178a-41db-9710-ebb584788cf3","Type":"ContainerStarted","Data":"ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed"} Jul 22 04:39:54.214768 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.214731 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:39:54.236285 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.236244 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" podStartSLOduration=1.421837208 podStartE2EDuration="6.23622967s" totalImagesPullingTime="4.814392462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:54.233338714 +0000 UTC m=+409.281606973" watchObservedRunningTime="2026-07-22 04:39:54.23622967 +0000 UTC m=+409.284497930" Jul 22 04:39:54.248040 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:54.247973 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" podStartSLOduration=1.486990335 podStartE2EDuration="4.247960014s" totalImagesPullingTime="2.760969679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:54.247535701 +0000 UTC m=+409.295803960" watchObservedRunningTime="2026-07-22 04:39:54.247960014 +0000 UTC m=+409.296228272" Jul 22 04:39:55.157488 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:55.157460 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z2sl6" Jul 22 04:39:58.174676 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:39:58.174641 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lmdvn" Jul 22 04:40:05.220860 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:05.220821 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:05.221292 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:05.220894 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:06.894116 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.894083 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz"] Jul 22 04:40:06.894579 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.894355 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" containerName="manager" containerID="cri-o://b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512" gracePeriod=2 Jul 22 04:40:06.912471 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.912436 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz"] Jul 22 04:40:06.925574 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:06.925535 2555 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"94931e26-a6e3-4d4f-a40b-3c07c0b31354\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-vslnz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:06.929638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.929614 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml"] Jul 22 04:40:06.929898 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.929844 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" podUID="317808c1-178a-41db-9710-ebb584788cf3" containerName="manager" containerID="cri-o://ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed" gracePeriod=2 Jul 22 04:40:06.932630 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:06.932587 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:06.937895 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.937870 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:06.938297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.938282 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" containerName="manager" Jul 22 04:40:06.938297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.938298 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" containerName="manager" Jul 22 04:40:06.938405 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.938395 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" containerName="manager" Jul 22 04:40:06.941238 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.941220 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:06.954123 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.954101 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml"] Jul 22 04:40:06.963706 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.963682 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:06.970299 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:06.970269 2555 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"317808c1-178a-41db-9710-ebb584788cf3\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-g4cml\": pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:06.972540 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:06.972514 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:06.972641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.972544 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms"] Jul 22 04:40:06.973053 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.973017 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="317808c1-178a-41db-9710-ebb584788cf3" containerName="manager" Jul 22 04:40:06.973053 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.973054 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="317808c1-178a-41db-9710-ebb584788cf3" containerName="manager" Jul 22 04:40:06.973184 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.973173 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="317808c1-178a-41db-9710-ebb584788cf3" containerName="manager" Jul 22 04:40:06.974486 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:06.974462 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:06.976316 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.976301 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:06.987139 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.987114 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms"] Jul 22 04:40:06.996051 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.995278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmgr\" (UniqueName: \"kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:06.996051 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.995341 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:06.996051 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:06.995413 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdbs\" (UniqueName: \"kubernetes.io/projected/4478a13d-7890-45a5-b1fc-2e6e975380a3-kube-api-access-wmdbs\") pod \"limitador-operator-controller-manager-859d7b78bd-8krms\" (UID: \"4478a13d-7890-45a5-b1fc-2e6e975380a3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:07.015274 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.015242 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.017167 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.017143 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.096330 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.096299 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmgr\" (UniqueName: \"kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:07.096471 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.096440 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:07.096628 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.096606 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdbs\" (UniqueName: \"kubernetes.io/projected/4478a13d-7890-45a5-b1fc-2e6e975380a3-kube-api-access-wmdbs\") pod \"limitador-operator-controller-manager-859d7b78bd-8krms\" (UID: \"4478a13d-7890-45a5-b1fc-2e6e975380a3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:07.096905 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.096882 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:07.114992 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.114280 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmgr\" (UniqueName: \"kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-f2r95\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:07.114992 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.114420 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdbs\" (UniqueName: \"kubernetes.io/projected/4478a13d-7890-45a5-b1fc-2e6e975380a3-kube-api-access-wmdbs\") pod \"limitador-operator-controller-manager-859d7b78bd-8krms\" (UID: \"4478a13d-7890-45a5-b1fc-2e6e975380a3\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:07.162102 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.162080 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.164357 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.164333 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.166382 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.166355 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.167430 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.167412 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.169700 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.169675 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.171499 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.171477 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.197596 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.197577 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume\") pod \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " Jul 22 04:40:07.197682 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.197638 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vww49\" (UniqueName: \"kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49\") pod \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\" (UID: \"94931e26-a6e3-4d4f-a40b-3c07c0b31354\") " Jul 22 04:40:07.197682 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.197673 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqsqv\" (UniqueName: \"kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv\") pod \"317808c1-178a-41db-9710-ebb584788cf3\" (UID: \"317808c1-178a-41db-9710-ebb584788cf3\") " Jul 22 04:40:07.197920 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.197898 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "94931e26-a6e3-4d4f-a40b-3c07c0b31354" (UID: "94931e26-a6e3-4d4f-a40b-3c07c0b31354"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:40:07.199587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.199567 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv" (OuterVolumeSpecName: "kube-api-access-bqsqv") pod "317808c1-178a-41db-9710-ebb584788cf3" (UID: "317808c1-178a-41db-9710-ebb584788cf3"). InnerVolumeSpecName "kube-api-access-bqsqv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:40:07.199691 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.199598 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49" (OuterVolumeSpecName: "kube-api-access-vww49") pod "94931e26-a6e3-4d4f-a40b-3c07c0b31354" (UID: "94931e26-a6e3-4d4f-a40b-3c07c0b31354"). InnerVolumeSpecName "kube-api-access-vww49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:40:07.280149 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.280118 2555 generic.go:356] "Generic (PLEG): container finished" podID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" containerID="b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512" exitCode=0 Jul 22 04:40:07.280245 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.280160 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.280245 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.280173 2555 scope.go:117] "RemoveContainer" containerID="b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512" Jul 22 04:40:07.281682 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.281659 2555 generic.go:356] "Generic (PLEG): container finished" podID="317808c1-178a-41db-9710-ebb584788cf3" containerID="ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed" exitCode=0 Jul 22 04:40:07.281767 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.281700 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.282733 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.282709 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.285050 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.285003 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.287165 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.287142 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.288350 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.288333 2555 scope.go:117] "RemoveContainer" containerID="b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512" Jul 22 04:40:07.288619 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.288601 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512\": container with ID starting with b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512 not found: ID does not exist" containerID="b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512" Jul 22 04:40:07.288665 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.288626 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512"} err="failed to get container status \"b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512\": rpc error: code = NotFound desc = could not find container \"b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512\": container with ID starting with b245514d931c42986ba9baa52b258bd49f26d546d62ef7033ca1c844747d6512 not found: ID does not exist" Jul 22 04:40:07.288665 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.288643 2555 scope.go:117] "RemoveContainer" containerID="ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed" Jul 22 04:40:07.289205 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.289178 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.294755 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.294733 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.297721 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.297124 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.298779 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.298751 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/94931e26-a6e3-4d4f-a40b-3c07c0b31354-extensions-socket-volume\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:07.298868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.298784 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vww49\" (UniqueName: \"kubernetes.io/projected/94931e26-a6e3-4d4f-a40b-3c07c0b31354-kube-api-access-vww49\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:07.298868 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.298800 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqsqv\" (UniqueName: \"kubernetes.io/projected/317808c1-178a-41db-9710-ebb584788cf3-kube-api-access-bqsqv\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:07.299521 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.299467 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-g4cml\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="317808c1-178a-41db-9710-ebb584788cf3" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-g4cml" Jul 22 04:40:07.299808 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.299733 2555 scope.go:117] "RemoveContainer" containerID="ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed" Jul 22 04:40:07.300154 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.300133 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed\": container with ID starting with ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed not found: ID does not exist" containerID="ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed" Jul 22 04:40:07.300307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.300162 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed"} err="failed to get container status \"ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed\": rpc error: code = NotFound desc = could not find container \"ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed\": container with ID starting with ce60e9c002e0b0a3ac887eace20e1a8e46502740d8be016c040fd8ff62e7d3ed not found: ID does not exist" Jul 22 04:40:07.301391 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:07.301370 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vslnz\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vslnz" Jul 22 04:40:07.338545 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.338521 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:07.346727 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.346702 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:07.442098 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.442069 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="317808c1-178a-41db-9710-ebb584788cf3" path="/var/lib/kubelet/pods/317808c1-178a-41db-9710-ebb584788cf3/volumes" Jul 22 04:40:07.442440 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.442424 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94931e26-a6e3-4d4f-a40b-3c07c0b31354" path="/var/lib/kubelet/pods/94931e26-a6e3-4d4f-a40b-3c07c0b31354/volumes" Jul 22 04:40:07.482694 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.482526 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:07.485113 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:07.485087 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a2f9f6_29bd_47ec_a25a_032e0faeddab.slice/crio-4e07508968f760764c6560fb13c205d3fda17250b3f353d98209fa0c07c76453 WatchSource:0}: Error finding container 4e07508968f760764c6560fb13c205d3fda17250b3f353d98209fa0c07c76453: Status 404 returned error can't find the container with id 4e07508968f760764c6560fb13c205d3fda17250b3f353d98209fa0c07c76453 Jul 22 04:40:07.498819 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:07.498794 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms"] Jul 22 04:40:07.502713 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:07.502694 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4478a13d_7890_45a5_b1fc_2e6e975380a3.slice/crio-731ed380c1446472ba5acfd341f564b44b926c45db248ffe2a54e0628635761a WatchSource:0}: Error finding container 731ed380c1446472ba5acfd341f564b44b926c45db248ffe2a54e0628635761a: Status 404 returned error can't find the container with id 731ed380c1446472ba5acfd341f564b44b926c45db248ffe2a54e0628635761a Jul 22 04:40:08.290835 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.290784 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" event={"ID":"4478a13d-7890-45a5-b1fc-2e6e975380a3","Type":"ContainerStarted","Data":"c59671982b948fee2ec0a555ec47def8ac0cb2ab768fe7b0698a46565ca1e194"} Jul 22 04:40:08.291278 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.290916 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:08.291278 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.290951 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" event={"ID":"4478a13d-7890-45a5-b1fc-2e6e975380a3","Type":"ContainerStarted","Data":"731ed380c1446472ba5acfd341f564b44b926c45db248ffe2a54e0628635761a"} Jul 22 04:40:08.292542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.292514 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" event={"ID":"f7a2f9f6-29bd-47ec-a25a-032e0faeddab","Type":"ContainerStarted","Data":"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d"} Jul 22 04:40:08.292607 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.292565 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" event={"ID":"f7a2f9f6-29bd-47ec-a25a-032e0faeddab","Type":"ContainerStarted","Data":"4e07508968f760764c6560fb13c205d3fda17250b3f353d98209fa0c07c76453"} Jul 22 04:40:08.292607 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.292586 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:08.310548 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.310497 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" podStartSLOduration=2.310480566 podStartE2EDuration="2.310480566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:08.309117116 +0000 UTC m=+423.357385375" watchObservedRunningTime="2026-07-22 04:40:08.310480566 +0000 UTC m=+423.358748902" Jul 22 04:40:08.326990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:08.326944 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" podStartSLOduration=2.326928604 podStartE2EDuration="2.326928604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:08.326237972 +0000 UTC m=+423.374506229" watchObservedRunningTime="2026-07-22 04:40:08.326928604 +0000 UTC m=+423.375196861" Jul 22 04:40:14.263860 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.263818 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dddc9f556-wqscb"] Jul 22 04:40:14.272401 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.272378 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.276109 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276085 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 04:40:14.276798 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276777 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vn4zg\"" Jul 22 04:40:14.276861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276843 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 04:40:14.276920 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276887 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 04:40:14.276972 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276924 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 04:40:14.276972 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.276959 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 04:40:14.285600 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.285573 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 04:40:14.286461 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.286436 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dddc9f556-wqscb"] Jul 22 04:40:14.358066 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358040 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-service-ca\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358169 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358074 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-oauth-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358169 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358100 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-trusted-ca-bundle\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358185 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358216 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358311 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358255 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9d6x\" (UniqueName: \"kubernetes.io/projected/63d326f7-ae10-4dc0-afec-3c4e2e398525-kube-api-access-t9d6x\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.358311 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.358297 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-oauth-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.458746 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.458713 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9d6x\" (UniqueName: \"kubernetes.io/projected/63d326f7-ae10-4dc0-afec-3c4e2e398525-kube-api-access-t9d6x\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.458861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.458776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-oauth-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.458861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.458805 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-service-ca\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.458861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.458822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-oauth-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.458861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.458844 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-trusted-ca-bundle\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459189 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459287 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459694 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459655 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-service-ca\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459694 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459679 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-oauth-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459867 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459845 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.459936 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.459868 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63d326f7-ae10-4dc0-afec-3c4e2e398525-trusted-ca-bundle\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.461539 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.461511 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-oauth-config\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.461740 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.461719 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/63d326f7-ae10-4dc0-afec-3c4e2e398525-console-serving-cert\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.466793 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.466767 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9d6x\" (UniqueName: \"kubernetes.io/projected/63d326f7-ae10-4dc0-afec-3c4e2e398525-kube-api-access-t9d6x\") pod \"console-6dddc9f556-wqscb\" (UID: \"63d326f7-ae10-4dc0-afec-3c4e2e398525\") " pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.583936 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.583886 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:14.710449 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:14.710426 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dddc9f556-wqscb"] Jul 22 04:40:14.712391 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:14.712369 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d326f7_ae10_4dc0_afec_3c4e2e398525.slice/crio-39f084ece5b5a60604f9dcc1da63a980bded2b017732013b96175cd3dbca96bc WatchSource:0}: Error finding container 39f084ece5b5a60604f9dcc1da63a980bded2b017732013b96175cd3dbca96bc: Status 404 returned error can't find the container with id 39f084ece5b5a60604f9dcc1da63a980bded2b017732013b96175cd3dbca96bc Jul 22 04:40:15.333107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:15.333061 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dddc9f556-wqscb" event={"ID":"63d326f7-ae10-4dc0-afec-3c4e2e398525","Type":"ContainerStarted","Data":"f2861891f021403d13c56484cd0434e3669d475940b37738d1bc30d71e31456e"} Jul 22 04:40:15.333556 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:15.333134 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dddc9f556-wqscb" event={"ID":"63d326f7-ae10-4dc0-afec-3c4e2e398525","Type":"ContainerStarted","Data":"39f084ece5b5a60604f9dcc1da63a980bded2b017732013b96175cd3dbca96bc"} Jul 22 04:40:15.351506 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:15.351124 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dddc9f556-wqscb" podStartSLOduration=1.351102215 podStartE2EDuration="1.351102215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:15.349777937 +0000 UTC m=+430.398046195" watchObservedRunningTime="2026-07-22 04:40:15.351102215 +0000 UTC m=+430.399370472" Jul 22 04:40:19.299529 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:19.299499 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:19.299934 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:19.299725 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8krms" Jul 22 04:40:23.118737 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.118706 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:23.119161 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.118916 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" podUID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" containerName="manager" containerID="cri-o://a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d" gracePeriod=10 Jul 22 04:40:23.327003 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.326974 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:40:23.330893 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.330871 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.342197 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.342082 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:40:23.362324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.362305 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:23.376927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.376871 2555 generic.go:356] "Generic (PLEG): container finished" podID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" containerID="a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d" exitCode=0 Jul 22 04:40:23.376927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.376906 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" Jul 22 04:40:23.377080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.376945 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" event={"ID":"f7a2f9f6-29bd-47ec-a25a-032e0faeddab","Type":"ContainerDied","Data":"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d"} Jul 22 04:40:23.377080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.376995 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95" event={"ID":"f7a2f9f6-29bd-47ec-a25a-032e0faeddab","Type":"ContainerDied","Data":"4e07508968f760764c6560fb13c205d3fda17250b3f353d98209fa0c07c76453"} Jul 22 04:40:23.377080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.377009 2555 scope.go:117] "RemoveContainer" containerID="a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d" Jul 22 04:40:23.389861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.389839 2555 scope.go:117] "RemoveContainer" containerID="a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d" Jul 22 04:40:23.390212 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:23.390179 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d\": container with ID starting with a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d not found: ID does not exist" containerID="a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d" Jul 22 04:40:23.390326 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.390216 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d"} err="failed to get container status \"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d\": rpc error: code = NotFound desc = could not find container \"a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d\": container with ID starting with a5c79f5ab100eb2986354ead207fb9e17b9f2003ff606fb7e425016fddd2578d not found: ID does not exist" Jul 22 04:40:23.421454 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.421431 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume\") pod \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " Jul 22 04:40:23.421523 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.421477 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrmgr\" (UniqueName: \"kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr\") pod \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\" (UID: \"f7a2f9f6-29bd-47ec-a25a-032e0faeddab\") " Jul 22 04:40:23.421573 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.421559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.421606 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.421593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfbmq\" (UniqueName: \"kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.421747 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.421727 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7a2f9f6-29bd-47ec-a25a-032e0faeddab" (UID: "f7a2f9f6-29bd-47ec-a25a-032e0faeddab"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:40:23.423379 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.423350 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr" (OuterVolumeSpecName: "kube-api-access-wrmgr") pod "f7a2f9f6-29bd-47ec-a25a-032e0faeddab" (UID: "f7a2f9f6-29bd-47ec-a25a-032e0faeddab"). InnerVolumeSpecName "kube-api-access-wrmgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:40:23.470324 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:23.470302 2555 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a2f9f6_29bd_47ec_a25a_032e0faeddab.slice\": RecentStats: unable to find data in memory cache]" Jul 22 04:40:23.522446 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.522420 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.522523 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.522453 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfbmq\" (UniqueName: \"kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.522523 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.522487 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-extensions-socket-volume\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:23.522523 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.522497 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrmgr\" (UniqueName: \"kubernetes.io/projected/f7a2f9f6-29bd-47ec-a25a-032e0faeddab-kube-api-access-wrmgr\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:23.522836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.522820 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.531239 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.531216 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfbmq\" (UniqueName: \"kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq\") pod \"kuadrant-operator-controller-manager-86878c769f-bl8qz\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.642105 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.642062 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:23.695874 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.695842 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:23.705498 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.705473 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-f2r95"] Jul 22 04:40:23.775423 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:23.775396 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:40:23.777748 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:23.777718 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc7d412_0c28_41af_8fc5_63ec4a470e7d.slice/crio-88505ebe31bcc7f65c3e22993efd0e495601b6780df0ccccea336e0ac97ea45c WatchSource:0}: Error finding container 88505ebe31bcc7f65c3e22993efd0e495601b6780df0ccccea336e0ac97ea45c: Status 404 returned error can't find the container with id 88505ebe31bcc7f65c3e22993efd0e495601b6780df0ccccea336e0ac97ea45c Jul 22 04:40:24.384844 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.384812 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" event={"ID":"3bc7d412-0c28-41af-8fc5-63ec4a470e7d","Type":"ContainerStarted","Data":"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca"} Jul 22 04:40:24.385265 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.384860 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" event={"ID":"3bc7d412-0c28-41af-8fc5-63ec4a470e7d","Type":"ContainerStarted","Data":"88505ebe31bcc7f65c3e22993efd0e495601b6780df0ccccea336e0ac97ea45c"} Jul 22 04:40:24.385265 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.384896 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:24.406932 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.406881 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" podStartSLOduration=1.40686666 podStartE2EDuration="1.40686666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:24.403696644 +0000 UTC m=+439.451964901" watchObservedRunningTime="2026-07-22 04:40:24.40686666 +0000 UTC m=+439.455134917" Jul 22 04:40:24.584709 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.584671 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:24.584841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.584729 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:24.589090 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:24.589064 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:25.394234 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:25.394210 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dddc9f556-wqscb" Jul 22 04:40:25.432008 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:25.431978 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" path="/var/lib/kubelet/pods/f7a2f9f6-29bd-47ec-a25a-032e0faeddab/volumes" Jul 22 04:40:35.392835 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:35.392759 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:40:39.438444 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.438407 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp"] Jul 22 04:40:39.438966 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.438944 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" containerName="manager" Jul 22 04:40:39.439072 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.438968 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" containerName="manager" Jul 22 04:40:39.439141 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.439094 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a2f9f6-29bd-47ec-a25a-032e0faeddab" containerName="manager" Jul 22 04:40:39.442406 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.442381 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.444707 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.444688 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-np5wz\"" Jul 22 04:40:39.454320 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.454298 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp"] Jul 22 04:40:39.538572 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538545 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538693 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538693 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538656 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538693 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538822 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538724 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538822 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538794 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538897 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538843 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538897 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.538991 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.538930 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rbk5\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-kube-api-access-6rbk5\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.639959 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.639935 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.639971 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640002 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640045 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640080 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640111 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rbk5\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-kube-api-access-6rbk5\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640175 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640212 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640415 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640473 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640490 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640640 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640578 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.640968 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.640940 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.642542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.642513 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.642645 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.642615 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.647308 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.647286 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.647492 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.647470 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rbk5\" (UniqueName: \"kubernetes.io/projected/00d98c58-f3ca-4f07-a2bd-c77963aefb13-kube-api-access-6rbk5\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jspkp\" (UID: \"00d98c58-f3ca-4f07-a2bd-c77963aefb13\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.752878 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.752852 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:39.882852 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.882823 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp"] Jul 22 04:40:39.884551 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:39.884523 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d98c58_f3ca_4f07_a2bd_c77963aefb13.slice/crio-032064fbbb995794251d15798dee3975ef85f86553cb963aa8ab3b2dee650256 WatchSource:0}: Error finding container 032064fbbb995794251d15798dee3975ef85f86553cb963aa8ab3b2dee650256: Status 404 returned error can't find the container with id 032064fbbb995794251d15798dee3975ef85f86553cb963aa8ab3b2dee650256 Jul 22 04:40:39.886697 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.886663 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:40:39.886752 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.886737 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:40:39.886790 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:39.886774 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:40:40.473773 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:40.473739 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" event={"ID":"00d98c58-f3ca-4f07-a2bd-c77963aefb13","Type":"ContainerStarted","Data":"84000c34f4bd17faf497e7af5d9c9b4192ef6d8a67ddcad2a28d87b00c1941e7"} Jul 22 04:40:40.474170 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:40.473790 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" event={"ID":"00d98c58-f3ca-4f07-a2bd-c77963aefb13","Type":"ContainerStarted","Data":"032064fbbb995794251d15798dee3975ef85f86553cb963aa8ab3b2dee650256"} Jul 22 04:40:40.505714 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:40.505664 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" podStartSLOduration=1.505635583 podStartE2EDuration="1.505635583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:40.504796296 +0000 UTC m=+455.553064554" watchObservedRunningTime="2026-07-22 04:40:40.505635583 +0000 UTC m=+455.553903896" Jul 22 04:40:40.753905 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:40.753873 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:40.758324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:40.758301 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:41.478447 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:41.478410 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:41.479442 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:41.479421 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:40:52.339107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.339070 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:40:52.343953 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.343931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:52.346943 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.346458 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5rmx\"" Jul 22 04:40:52.349107 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.349087 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:40:52.441784 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.441755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnmhf\" (UniqueName: \"kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf\") pod \"authorino-58cd955675-qcnh8\" (UID: \"9704eea6-bb70-48d3-b6d2-7abab4783fb6\") " pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:52.543201 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.543167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnmhf\" (UniqueName: \"kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf\") pod \"authorino-58cd955675-qcnh8\" (UID: \"9704eea6-bb70-48d3-b6d2-7abab4783fb6\") " pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:52.552339 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.552317 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnmhf\" (UniqueName: \"kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf\") pod \"authorino-58cd955675-qcnh8\" (UID: \"9704eea6-bb70-48d3-b6d2-7abab4783fb6\") " pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:52.654643 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.654579 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:52.777605 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:52.777583 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:40:52.779416 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:40:52.779384 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9704eea6_bb70_48d3_b6d2_7abab4783fb6.slice/crio-02ea3cc1fa51059dc4b073d57dd19c505ec02ca0825b2e99ff51895a49005889 WatchSource:0}: Error finding container 02ea3cc1fa51059dc4b073d57dd19c505ec02ca0825b2e99ff51895a49005889: Status 404 returned error can't find the container with id 02ea3cc1fa51059dc4b073d57dd19c505ec02ca0825b2e99ff51895a49005889 Jul 22 04:40:53.536576 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:53.536537 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qcnh8" event={"ID":"9704eea6-bb70-48d3-b6d2-7abab4783fb6","Type":"ContainerStarted","Data":"02ea3cc1fa51059dc4b073d57dd19c505ec02ca0825b2e99ff51895a49005889"} Jul 22 04:40:56.554044 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:56.553982 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qcnh8" event={"ID":"9704eea6-bb70-48d3-b6d2-7abab4783fb6","Type":"ContainerStarted","Data":"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088"} Jul 22 04:40:56.569551 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:56.569496 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-qcnh8" podStartSLOduration=1.416272696 podStartE2EDuration="4.569479521s" totalImagesPullingTime="3.153206825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:56.567793704 +0000 UTC m=+471.616061962" watchObservedRunningTime="2026-07-22 04:40:56.569479521 +0000 UTC m=+471.617747778" Jul 22 04:40:57.431850 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:57.431816 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:40:58.563809 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:58.563751 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-qcnh8" podUID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" containerName="authorino" containerID="cri-o://bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088" gracePeriod=30 Jul 22 04:40:58.810666 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:58.810643 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:58.893970 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:58.893910 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bnmhf\" (UniqueName: \"kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf\") pod \"9704eea6-bb70-48d3-b6d2-7abab4783fb6\" (UID: \"9704eea6-bb70-48d3-b6d2-7abab4783fb6\") " Jul 22 04:40:58.895850 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:58.895830 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf" (OuterVolumeSpecName: "kube-api-access-bnmhf") pod "9704eea6-bb70-48d3-b6d2-7abab4783fb6" (UID: "9704eea6-bb70-48d3-b6d2-7abab4783fb6"). InnerVolumeSpecName "kube-api-access-bnmhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:40:58.994822 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:58.994801 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnmhf\" (UniqueName: \"kubernetes.io/projected/9704eea6-bb70-48d3-b6d2-7abab4783fb6-kube-api-access-bnmhf\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:40:59.569912 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.569883 2555 generic.go:356] "Generic (PLEG): container finished" podID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" containerID="bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088" exitCode=0 Jul 22 04:40:59.570356 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.569931 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-qcnh8" Jul 22 04:40:59.570356 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.569951 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qcnh8" event={"ID":"9704eea6-bb70-48d3-b6d2-7abab4783fb6","Type":"ContainerDied","Data":"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088"} Jul 22 04:40:59.570356 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.569980 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-qcnh8" event={"ID":"9704eea6-bb70-48d3-b6d2-7abab4783fb6","Type":"ContainerDied","Data":"02ea3cc1fa51059dc4b073d57dd19c505ec02ca0825b2e99ff51895a49005889"} Jul 22 04:40:59.570356 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.569996 2555 scope.go:117] "RemoveContainer" containerID="bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088" Jul 22 04:40:59.578145 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.578119 2555 scope.go:117] "RemoveContainer" containerID="bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088" Jul 22 04:40:59.578424 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:40:59.578403 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088\": container with ID starting with bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088 not found: ID does not exist" containerID="bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088" Jul 22 04:40:59.578488 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.578432 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088"} err="failed to get container status \"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088\": rpc error: code = NotFound desc = could not find container \"bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088\": container with ID starting with bc483900699c87207ab1986ee9c184c5c3e1cd0e6385183bab3bfa50adf0c088 not found: ID does not exist" Jul 22 04:40:59.584293 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.584270 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:40:59.588109 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:40:59.588091 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-qcnh8"] Jul 22 04:41:01.430322 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:01.430291 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" path="/var/lib/kubelet/pods/9704eea6-bb70-48d3-b6d2-7abab4783fb6/volumes" Jul 22 04:41:21.037147 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.037113 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48l74"] Jul 22 04:41:21.037564 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.037503 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" containerName="authorino" Jul 22 04:41:21.037564 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.037516 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" containerName="authorino" Jul 22 04:41:21.037642 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.037574 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9704eea6-bb70-48d3-b6d2-7abab4783fb6" containerName="authorino" Jul 22 04:41:21.041989 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.041967 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.044387 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.044363 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m5rmx\"" Jul 22 04:41:21.046990 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.046965 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48l74"] Jul 22 04:41:21.175474 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.175448 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbjx7\" (UniqueName: \"kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7\") pod \"authorino-5d4b67c446-48l74\" (UID: \"09ae5cce-83ed-4dae-9352-ba6a90caaf94\") " pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.267704 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.267673 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48l74"] Jul 22 04:41:21.267945 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:41:21.267923 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xbjx7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-48l74" podUID="09ae5cce-83ed-4dae-9352-ba6a90caaf94" Jul 22 04:41:21.275949 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.275925 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbjx7\" (UniqueName: \"kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7\") pod \"authorino-5d4b67c446-48l74\" (UID: \"09ae5cce-83ed-4dae-9352-ba6a90caaf94\") " pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.289446 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.289390 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:21.291095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.291067 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbjx7\" (UniqueName: \"kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7\") pod \"authorino-5d4b67c446-48l74\" (UID: \"09ae5cce-83ed-4dae-9352-ba6a90caaf94\") " pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.291700 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.291686 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:21.298535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.298515 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:21.477757 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.477721 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc67m\" (UniqueName: \"kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m\") pod \"authorino-66d479f444-r4nsv\" (UID: \"f9faf19f-6a7e-4e54-b989-196c085faa24\") " pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:21.579315 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.579223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc67m\" (UniqueName: \"kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m\") pod \"authorino-66d479f444-r4nsv\" (UID: \"f9faf19f-6a7e-4e54-b989-196c085faa24\") " pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:21.586945 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.586912 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc67m\" (UniqueName: \"kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m\") pod \"authorino-66d479f444-r4nsv\" (UID: \"f9faf19f-6a7e-4e54-b989-196c085faa24\") " pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:21.609419 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.609391 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:21.683110 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.683081 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.689116 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.689092 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:21.737985 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.737962 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:21.740337 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:41:21.740313 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9faf19f_6a7e_4e54_b989_196c085faa24.slice/crio-f46756a1e62e88eff39bad170a8d1f09c2f7daa331cc5365a058612b251ef456 WatchSource:0}: Error finding container f46756a1e62e88eff39bad170a8d1f09c2f7daa331cc5365a058612b251ef456: Status 404 returned error can't find the container with id f46756a1e62e88eff39bad170a8d1f09c2f7daa331cc5365a058612b251ef456 Jul 22 04:41:21.882114 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.882042 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xbjx7\" (UniqueName: \"kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7\") pod \"09ae5cce-83ed-4dae-9352-ba6a90caaf94\" (UID: \"09ae5cce-83ed-4dae-9352-ba6a90caaf94\") " Jul 22 04:41:21.884048 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.884010 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7" (OuterVolumeSpecName: "kube-api-access-xbjx7") pod "09ae5cce-83ed-4dae-9352-ba6a90caaf94" (UID: "09ae5cce-83ed-4dae-9352-ba6a90caaf94"). InnerVolumeSpecName "kube-api-access-xbjx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:41:21.982913 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:21.982889 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbjx7\" (UniqueName: \"kubernetes.io/projected/09ae5cce-83ed-4dae-9352-ba6a90caaf94-kube-api-access-xbjx7\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:41:22.696268 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.696240 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-48l74" Jul 22 04:41:22.696268 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.696235 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d479f444-r4nsv" event={"ID":"f9faf19f-6a7e-4e54-b989-196c085faa24","Type":"ContainerStarted","Data":"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed"} Jul 22 04:41:22.696838 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.696289 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d479f444-r4nsv" event={"ID":"f9faf19f-6a7e-4e54-b989-196c085faa24","Type":"ContainerStarted","Data":"f46756a1e62e88eff39bad170a8d1f09c2f7daa331cc5365a058612b251ef456"} Jul 22 04:41:22.713015 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.712964 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66d479f444-r4nsv" podStartSLOduration=1.319106701 podStartE2EDuration="1.712948317s" totalImagesPullingTime="393.841616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:41:22.712145823 +0000 UTC m=+497.760414084" watchObservedRunningTime="2026-07-22 04:41:22.712948317 +0000 UTC m=+497.761216576" Jul 22 04:41:22.734910 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.734846 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48l74"] Jul 22 04:41:22.740555 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.740529 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-48l74"] Jul 22 04:41:22.945978 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.945944 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:41:22.949399 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.949348 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:22.951597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.951574 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 04:41:22.955937 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.955915 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:41:22.990417 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.990388 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whq9\" (UniqueName: \"kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:22.990508 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:22.990449 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.091733 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.091704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whq9\" (UniqueName: \"kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.091841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.091790 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.094218 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.094199 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.098717 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.098698 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whq9\" (UniqueName: \"kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9\") pod \"authorino-7d79b59f64-dgx5x\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.259942 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.259918 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:41:23.392836 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.392811 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:41:23.394960 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:41:23.394932 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9665b667_573a_49a2_b86c_cff04ac1b151.slice/crio-40a03f379b91f1dc1359dc21eb306aea3aa5d8266986a53a93b1490307ca9909 WatchSource:0}: Error finding container 40a03f379b91f1dc1359dc21eb306aea3aa5d8266986a53a93b1490307ca9909: Status 404 returned error can't find the container with id 40a03f379b91f1dc1359dc21eb306aea3aa5d8266986a53a93b1490307ca9909 Jul 22 04:41:23.430583 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.430556 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ae5cce-83ed-4dae-9352-ba6a90caaf94" path="/var/lib/kubelet/pods/09ae5cce-83ed-4dae-9352-ba6a90caaf94/volumes" Jul 22 04:41:23.702215 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:23.702149 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" event={"ID":"9665b667-573a-49a2-b86c-cff04ac1b151","Type":"ContainerStarted","Data":"40a03f379b91f1dc1359dc21eb306aea3aa5d8266986a53a93b1490307ca9909"} Jul 22 04:41:24.708343 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:24.708307 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" event={"ID":"9665b667-573a-49a2-b86c-cff04ac1b151","Type":"ContainerStarted","Data":"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28"} Jul 22 04:41:24.722913 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:24.722869 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" podStartSLOduration=2.25106787 podStartE2EDuration="2.722852192s" totalImagesPullingTime="471.784322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:41:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:41:24.722343003 +0000 UTC m=+499.770611263" watchObservedRunningTime="2026-07-22 04:41:24.722852192 +0000 UTC m=+499.771120475" Jul 22 04:41:24.749872 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:24.749835 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:24.750044 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:24.750004 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66d479f444-r4nsv" podUID="f9faf19f-6a7e-4e54-b989-196c085faa24" containerName="authorino" containerID="cri-o://a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed" gracePeriod=30 Jul 22 04:41:25.027534 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.027508 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:25.210419 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.210394 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc67m\" (UniqueName: \"kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m\") pod \"f9faf19f-6a7e-4e54-b989-196c085faa24\" (UID: \"f9faf19f-6a7e-4e54-b989-196c085faa24\") " Jul 22 04:41:25.212445 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.212418 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m" (OuterVolumeSpecName: "kube-api-access-wc67m") pod "f9faf19f-6a7e-4e54-b989-196c085faa24" (UID: "f9faf19f-6a7e-4e54-b989-196c085faa24"). InnerVolumeSpecName "kube-api-access-wc67m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:41:25.311457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.311393 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc67m\" (UniqueName: \"kubernetes.io/projected/f9faf19f-6a7e-4e54-b989-196c085faa24-kube-api-access-wc67m\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:41:25.714641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.714583 2555 generic.go:356] "Generic (PLEG): container finished" podID="f9faf19f-6a7e-4e54-b989-196c085faa24" containerID="a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed" exitCode=0 Jul 22 04:41:25.714641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.714623 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66d479f444-r4nsv" Jul 22 04:41:25.714641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.714633 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d479f444-r4nsv" event={"ID":"f9faf19f-6a7e-4e54-b989-196c085faa24","Type":"ContainerDied","Data":"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed"} Jul 22 04:41:25.715055 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.714676 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66d479f444-r4nsv" event={"ID":"f9faf19f-6a7e-4e54-b989-196c085faa24","Type":"ContainerDied","Data":"f46756a1e62e88eff39bad170a8d1f09c2f7daa331cc5365a058612b251ef456"} Jul 22 04:41:25.715055 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.714700 2555 scope.go:117] "RemoveContainer" containerID="a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed" Jul 22 04:41:25.723364 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.723345 2555 scope.go:117] "RemoveContainer" containerID="a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed" Jul 22 04:41:25.723596 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:41:25.723579 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed\": container with ID starting with a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed not found: ID does not exist" containerID="a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed" Jul 22 04:41:25.723649 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.723605 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed"} err="failed to get container status \"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed\": rpc error: code = NotFound desc = could not find container \"a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed\": container with ID starting with a7c4d6ada99c3ad02185672e173474464f3d76902791291f1e45ecae8d3ac4ed not found: ID does not exist" Jul 22 04:41:25.729727 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.729705 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:25.736353 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:25.736323 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66d479f444-r4nsv"] Jul 22 04:41:27.431908 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.431874 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9faf19f-6a7e-4e54-b989-196c085faa24" path="/var/lib/kubelet/pods/f9faf19f-6a7e-4e54-b989-196c085faa24/volumes" Jul 22 04:41:27.588927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.588886 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:41:27.589357 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.589339 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9faf19f-6a7e-4e54-b989-196c085faa24" containerName="authorino" Jul 22 04:41:27.589439 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.589359 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9faf19f-6a7e-4e54-b989-196c085faa24" containerName="authorino" Jul 22 04:41:27.589509 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.589497 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9faf19f-6a7e-4e54-b989-196c085faa24" containerName="authorino" Jul 22 04:41:27.594103 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.594083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.596515 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.596494 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 04:41:27.596610 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.596561 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 04:41:27.596657 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.596559 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-chh6j\"" Jul 22 04:41:27.602477 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.602453 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:41:27.626961 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.626940 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlmjc\" (UniqueName: \"kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.627062 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.626995 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.728009 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.727927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlmjc\" (UniqueName: \"kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.728009 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.727981 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.730370 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.730346 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.735557 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.735533 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlmjc\" (UniqueName: \"kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc\") pod \"maas-controller-865d8b7674-v7646\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:27.904577 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:27.904542 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:28.034740 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:28.034714 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:41:28.038948 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:41:28.038906 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf427e4ee_9121_4411_ac25_f09af5ecd5a1.slice/crio-f19c4a807aea4c6e7ec0001ae39a518b914277548a7eca7c75dfc8a0cf6181a5 WatchSource:0}: Error finding container f19c4a807aea4c6e7ec0001ae39a518b914277548a7eca7c75dfc8a0cf6181a5: Status 404 returned error can't find the container with id f19c4a807aea4c6e7ec0001ae39a518b914277548a7eca7c75dfc8a0cf6181a5 Jul 22 04:41:28.736258 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:28.736222 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-v7646" event={"ID":"f427e4ee-9121-4411-ac25-f09af5ecd5a1","Type":"ContainerStarted","Data":"f19c4a807aea4c6e7ec0001ae39a518b914277548a7eca7c75dfc8a0cf6181a5"} Jul 22 04:41:30.748248 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:30.748212 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-v7646" event={"ID":"f427e4ee-9121-4411-ac25-f09af5ecd5a1","Type":"ContainerStarted","Data":"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c"} Jul 22 04:41:30.748596 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:30.748311 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:41:30.765174 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:30.765123 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865d8b7674-v7646" podStartSLOduration=1.422456898 podStartE2EDuration="3.765107605s" totalImagesPullingTime="2.342650707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:41:30.763983494 +0000 UTC m=+505.812251753" watchObservedRunningTime="2026-07-22 04:41:30.765107605 +0000 UTC m=+505.813375864" Jul 22 04:41:41.755883 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:41:41.755851 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:42:10.953411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.953324 2555 generic.go:356] "Generic (PLEG): container finished" podID="00d98c58-f3ca-4f07-a2bd-c77963aefb13" containerID="84000c34f4bd17faf497e7af5d9c9b4192ef6d8a67ddcad2a28d87b00c1941e7" exitCode=0 Jul 22 04:42:10.953411 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.953395 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" event={"ID":"00d98c58-f3ca-4f07-a2bd-c77963aefb13","Type":"ContainerDied","Data":"84000c34f4bd17faf497e7af5d9c9b4192ef6d8a67ddcad2a28d87b00c1941e7"} Jul 22 04:42:10.953832 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.953792 2555 scope.go:117] "RemoveContainer" containerID="84000c34f4bd17faf497e7af5d9c9b4192ef6d8a67ddcad2a28d87b00c1941e7" Jul 22 04:42:10.954798 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.954770 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:42:10.954881 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.954841 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:42:10.954881 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:10.954877 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 04:42:11.478776 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:11.478744 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:42:11.960983 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:11.960943 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" event={"ID":"00d98c58-f3ca-4f07-a2bd-c77963aefb13","Type":"ContainerStarted","Data":"107cb93f1439967941706ea4d7b6fefa9fa6ac6eaa2f3016fd61b6aa0579429e"} Jul 22 04:42:12.753659 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:12.753619 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:42:12.754537 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:12.754513 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jspkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 04:42:12.754647 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:12.754567 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" podUID="00d98c58-f3ca-4f07-a2bd-c77963aefb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 04:42:13.753582 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:13.753543 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jspkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 04:42:13.754067 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:13.753607 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" podUID="00d98c58-f3ca-4f07-a2bd-c77963aefb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 04:42:14.753167 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:14.753132 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jspkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 04:42:14.753326 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:14.753187 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" podUID="00d98c58-f3ca-4f07-a2bd-c77963aefb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 04:42:15.753551 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:15.753506 2555 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jspkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 22 04:42:15.753952 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:15.753598 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" podUID="00d98c58-f3ca-4f07-a2bd-c77963aefb13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 22 04:42:16.757265 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:16.757236 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:42:16.757763 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:16.757562 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:42:16.758338 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:16.758313 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jspkp" Jul 22 04:42:44.358134 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.358074 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js"] Jul 22 04:42:44.370448 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.370419 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.371360 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.371332 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js"] Jul 22 04:42:44.373049 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.373006 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 04:42:44.373212 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.373193 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6ntgm\"" Jul 22 04:42:44.373286 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.373220 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 04:42:44.374167 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.374142 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 04:42:44.493133 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493095 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959749be-0866-43ed-af76-47966b9d5428-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.493317 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493159 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b4mx\" (UniqueName: \"kubernetes.io/projected/959749be-0866-43ed-af76-47966b9d5428-kube-api-access-5b4mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.493317 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493305 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.493446 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493352 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.493446 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.493551 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.493516 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594222 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959749be-0866-43ed-af76-47966b9d5428-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594246 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b4mx\" (UniqueName: \"kubernetes.io/projected/959749be-0866-43ed-af76-47966b9d5428-kube-api-access-5b4mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594272 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594294 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594420 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594337 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594796 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594771 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594873 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594820 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.594873 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.594840 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.596556 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.596534 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/959749be-0866-43ed-af76-47966b9d5428-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.596733 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.596712 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/959749be-0866-43ed-af76-47966b9d5428-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.601518 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.601494 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b4mx\" (UniqueName: \"kubernetes.io/projected/959749be-0866-43ed-af76-47966b9d5428-kube-api-access-5b4mx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xh6js\" (UID: \"959749be-0866-43ed-af76-47966b9d5428\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.683065 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.682957 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:44.819081 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:44.819055 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js"] Jul 22 04:42:44.821453 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:42:44.821421 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod959749be_0866_43ed_af76_47966b9d5428.slice/crio-f6c021fbfbe3fb299b749b505e460729e131d4d9380f888115ebc68f69644bf6 WatchSource:0}: Error finding container f6c021fbfbe3fb299b749b505e460729e131d4d9380f888115ebc68f69644bf6: Status 404 returned error can't find the container with id f6c021fbfbe3fb299b749b505e460729e131d4d9380f888115ebc68f69644bf6 Jul 22 04:42:45.129214 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:45.129127 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" event={"ID":"959749be-0866-43ed-af76-47966b9d5428","Type":"ContainerStarted","Data":"f6c021fbfbe3fb299b749b505e460729e131d4d9380f888115ebc68f69644bf6"} Jul 22 04:42:46.136457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:46.136411 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" event={"ID":"959749be-0866-43ed-af76-47966b9d5428","Type":"ContainerStarted","Data":"2904a7fb12dcfa4f47965664b62bc46890e42e8d2589a6256892f318fddd329b"} Jul 22 04:42:46.163929 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:46.163864 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" podStartSLOduration=1.001344244 podStartE2EDuration="2.16384734s" totalImagesPullingTime="1.162503096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:42:46.161187449 +0000 UTC m=+581.209455709" watchObservedRunningTime="2026-07-22 04:42:46.16384734 +0000 UTC m=+581.212115599" Jul 22 04:42:54.683691 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:54.683657 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:54.684163 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:54.683729 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:54.696185 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:54.696163 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:42:55.195377 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:42:55.195348 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xh6js" Jul 22 04:43:05.351462 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:05.351428 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:43:05.352861 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:05.352838 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:43:09.569977 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.569944 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q"] Jul 22 04:43:09.574297 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.574277 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.576686 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.576668 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 22 04:43:09.582705 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.582687 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q"] Jul 22 04:43:09.616851 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.616826 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.616939 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.616854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95ccr\" (UniqueName: \"kubernetes.io/projected/4a75524c-f539-4ffa-b3a7-507c49e2c277-kube-api-access-95ccr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.616939 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.616875 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a75524c-f539-4ffa-b3a7-507c49e2c277-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.617011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.616948 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.617011 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.616999 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.617102 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.617049 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718156 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718249 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95ccr\" (UniqueName: \"kubernetes.io/projected/4a75524c-f539-4ffa-b3a7-507c49e2c277-kube-api-access-95ccr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718324 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a75524c-f539-4ffa-b3a7-507c49e2c277-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718635 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718611 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718697 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718659 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.718745 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.718711 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.720379 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.720350 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a75524c-f539-4ffa-b3a7-507c49e2c277-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.720599 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.720578 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a75524c-f539-4ffa-b3a7-507c49e2c277-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.725204 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.725183 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95ccr\" (UniqueName: \"kubernetes.io/projected/4a75524c-f539-4ffa-b3a7-507c49e2c277-kube-api-access-95ccr\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njq9q\" (UID: \"4a75524c-f539-4ffa-b3a7-507c49e2c277\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:09.884684 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:09.884622 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:10.230213 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:10.230185 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q"] Jul 22 04:43:10.231391 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:43:10.231371 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a75524c_f539_4ffa_b3a7_507c49e2c277.slice/crio-913a7d6c3c9d926ebad1e8f7ef4590c959131f0a17d946aefacd83d24779a986 WatchSource:0}: Error finding container 913a7d6c3c9d926ebad1e8f7ef4590c959131f0a17d946aefacd83d24779a986: Status 404 returned error can't find the container with id 913a7d6c3c9d926ebad1e8f7ef4590c959131f0a17d946aefacd83d24779a986 Jul 22 04:43:10.232820 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:10.232800 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:43:10.254782 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:10.254743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" event={"ID":"4a75524c-f539-4ffa-b3a7-507c49e2c277","Type":"ContainerStarted","Data":"913a7d6c3c9d926ebad1e8f7ef4590c959131f0a17d946aefacd83d24779a986"} Jul 22 04:43:11.261321 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:11.261284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" event={"ID":"4a75524c-f539-4ffa-b3a7-507c49e2c277","Type":"ContainerStarted","Data":"09b0b1adfa07c25ef0a96c566ce5b40e8c101b2ae9d8ea04562147da590cbe20"} Jul 22 04:43:11.278148 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:11.278092 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" podStartSLOduration=2.098757589 podStartE2EDuration="2.278077749s" totalImagesPullingTime="179.32016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:43:11.276793704 +0000 UTC m=+606.325061963" watchObservedRunningTime="2026-07-22 04:43:11.278077749 +0000 UTC m=+606.326346007" Jul 22 04:43:19.885262 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:19.885231 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:19.885720 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:19.885289 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:19.898290 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:19.898264 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:43:20.320199 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:43:20.320170 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njq9q" Jul 22 04:45:36.672737 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.672704 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ff5b95d65-bkbxh"] Jul 22 04:45:36.676155 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.676133 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.686080 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.686056 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ff5b95d65-bkbxh"] Jul 22 04:45:36.763129 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.763104 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c440fc8-81c9-4d93-9958-e5684bba7db4-tls-cert\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.763254 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.763140 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rgmn\" (UniqueName: \"kubernetes.io/projected/2c440fc8-81c9-4d93-9958-e5684bba7db4-kube-api-access-6rgmn\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.863748 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.863721 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c440fc8-81c9-4d93-9958-e5684bba7db4-tls-cert\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.863848 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.863758 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rgmn\" (UniqueName: \"kubernetes.io/projected/2c440fc8-81c9-4d93-9958-e5684bba7db4-kube-api-access-6rgmn\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.865997 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.865977 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2c440fc8-81c9-4d93-9958-e5684bba7db4-tls-cert\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.870876 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.870856 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rgmn\" (UniqueName: \"kubernetes.io/projected/2c440fc8-81c9-4d93-9958-e5684bba7db4-kube-api-access-6rgmn\") pod \"authorino-5ff5b95d65-bkbxh\" (UID: \"2c440fc8-81c9-4d93-9958-e5684bba7db4\") " pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:36.986227 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:36.986179 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" Jul 22 04:45:37.116982 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:37.116956 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ff5b95d65-bkbxh"] Jul 22 04:45:37.119399 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:45:37.119368 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c440fc8_81c9_4d93_9958_e5684bba7db4.slice/crio-2c34a28be227daf71cc6c5acd9d2eef64bf86494b62a7c04af9d04c8e85342f2 WatchSource:0}: Error finding container 2c34a28be227daf71cc6c5acd9d2eef64bf86494b62a7c04af9d04c8e85342f2: Status 404 returned error can't find the container with id 2c34a28be227daf71cc6c5acd9d2eef64bf86494b62a7c04af9d04c8e85342f2 Jul 22 04:45:37.989585 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:37.989490 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" event={"ID":"2c440fc8-81c9-4d93-9958-e5684bba7db4","Type":"ContainerStarted","Data":"a9c91ce58fac250981f74da3e59d4e5c02be06b59848d29bb80f8ad91149dfe8"} Jul 22 04:45:37.989585 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:37.989549 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" event={"ID":"2c440fc8-81c9-4d93-9958-e5684bba7db4","Type":"ContainerStarted","Data":"2c34a28be227daf71cc6c5acd9d2eef64bf86494b62a7c04af9d04c8e85342f2"} Jul 22 04:45:38.006045 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.005968 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5ff5b95d65-bkbxh" podStartSLOduration=1.5412472529999999 podStartE2EDuration="2.005950667s" totalImagesPullingTime="464.703414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:45:38.004091134 +0000 UTC m=+753.052359395" watchObservedRunningTime="2026-07-22 04:45:38.005950667 +0000 UTC m=+753.054218925" Jul 22 04:45:38.027673 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.027638 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:45:38.027950 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.027919 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" podUID="9665b667-573a-49a2-b86c-cff04ac1b151" containerName="authorino" containerID="cri-o://333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28" gracePeriod=30 Jul 22 04:45:38.316776 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.316749 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:45:38.380196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.380160 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7whq9\" (UniqueName: \"kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9\") pod \"9665b667-573a-49a2-b86c-cff04ac1b151\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " Jul 22 04:45:38.380196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.380199 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert\") pod \"9665b667-573a-49a2-b86c-cff04ac1b151\" (UID: \"9665b667-573a-49a2-b86c-cff04ac1b151\") " Jul 22 04:45:38.382318 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.382287 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9" (OuterVolumeSpecName: "kube-api-access-7whq9") pod "9665b667-573a-49a2-b86c-cff04ac1b151" (UID: "9665b667-573a-49a2-b86c-cff04ac1b151"). InnerVolumeSpecName "kube-api-access-7whq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:45:38.391111 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.391087 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9665b667-573a-49a2-b86c-cff04ac1b151" (UID: "9665b667-573a-49a2-b86c-cff04ac1b151"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:45:38.481958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.481927 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7whq9\" (UniqueName: \"kubernetes.io/projected/9665b667-573a-49a2-b86c-cff04ac1b151-kube-api-access-7whq9\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:45:38.481958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.481955 2555 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9665b667-573a-49a2-b86c-cff04ac1b151-tls-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:45:38.996598 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.996561 2555 generic.go:356] "Generic (PLEG): container finished" podID="9665b667-573a-49a2-b86c-cff04ac1b151" containerID="333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28" exitCode=0 Jul 22 04:45:38.997086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.996616 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" Jul 22 04:45:38.997086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.996637 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" event={"ID":"9665b667-573a-49a2-b86c-cff04ac1b151","Type":"ContainerDied","Data":"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28"} Jul 22 04:45:38.997086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.996686 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d79b59f64-dgx5x" event={"ID":"9665b667-573a-49a2-b86c-cff04ac1b151","Type":"ContainerDied","Data":"40a03f379b91f1dc1359dc21eb306aea3aa5d8266986a53a93b1490307ca9909"} Jul 22 04:45:38.997086 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:38.996702 2555 scope.go:117] "RemoveContainer" containerID="333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28" Jul 22 04:45:39.007207 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:39.007125 2555 scope.go:117] "RemoveContainer" containerID="333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28" Jul 22 04:45:39.007531 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:45:39.007506 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28\": container with ID starting with 333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28 not found: ID does not exist" containerID="333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28" Jul 22 04:45:39.007608 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:39.007545 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28"} err="failed to get container status \"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28\": rpc error: code = NotFound desc = could not find container \"333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28\": container with ID starting with 333253c23888a037d9efe42be8edd2baf4368c55a8ea1412e942b034f3bb4f28 not found: ID does not exist" Jul 22 04:45:39.022603 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:39.022578 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:45:39.030932 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:39.030910 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d79b59f64-dgx5x"] Jul 22 04:45:39.431679 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:45:39.431646 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9665b667-573a-49a2-b86c-cff04ac1b151" path="/var/lib/kubelet/pods/9665b667-573a-49a2-b86c-cff04ac1b151/volumes" Jul 22 04:46:25.192181 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.192143 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:46:25.192640 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.192382 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-865d8b7674-v7646" podUID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" containerName="manager" containerID="cri-o://e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c" gracePeriod=30 Jul 22 04:46:25.445779 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.445733 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:46:25.475117 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.475093 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert\") pod \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " Jul 22 04:46:25.475238 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.475163 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlmjc\" (UniqueName: \"kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc\") pod \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\" (UID: \"f427e4ee-9121-4411-ac25-f09af5ecd5a1\") " Jul 22 04:46:25.477422 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.477397 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc" (OuterVolumeSpecName: "kube-api-access-xlmjc") pod "f427e4ee-9121-4411-ac25-f09af5ecd5a1" (UID: "f427e4ee-9121-4411-ac25-f09af5ecd5a1"). InnerVolumeSpecName "kube-api-access-xlmjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:46:25.477535 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.477510 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f427e4ee-9121-4411-ac25-f09af5ecd5a1" (UID: "f427e4ee-9121-4411-ac25-f09af5ecd5a1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:46:25.576455 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.576430 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlmjc\" (UniqueName: \"kubernetes.io/projected/f427e4ee-9121-4411-ac25-f09af5ecd5a1-kube-api-access-xlmjc\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:46:25.576570 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:25.576457 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f427e4ee-9121-4411-ac25-f09af5ecd5a1-webhook-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:46:26.260196 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.260157 2555 generic.go:356] "Generic (PLEG): container finished" podID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" containerID="e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c" exitCode=0 Jul 22 04:46:26.260590 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.260211 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-v7646" Jul 22 04:46:26.260590 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.260238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-v7646" event={"ID":"f427e4ee-9121-4411-ac25-f09af5ecd5a1","Type":"ContainerDied","Data":"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c"} Jul 22 04:46:26.260590 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.260313 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-v7646" event={"ID":"f427e4ee-9121-4411-ac25-f09af5ecd5a1","Type":"ContainerDied","Data":"f19c4a807aea4c6e7ec0001ae39a518b914277548a7eca7c75dfc8a0cf6181a5"} Jul 22 04:46:26.260590 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.260338 2555 scope.go:117] "RemoveContainer" containerID="e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c" Jul 22 04:46:26.269502 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.269484 2555 scope.go:117] "RemoveContainer" containerID="e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c" Jul 22 04:46:26.269734 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:46:26.269718 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c\": container with ID starting with e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c not found: ID does not exist" containerID="e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c" Jul 22 04:46:26.269780 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.269741 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c"} err="failed to get container status \"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c\": rpc error: code = NotFound desc = could not find container \"e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c\": container with ID starting with e2f00886c045dff853f6c98370b92c7dea4509a07071206d375a31c24e67108c not found: ID does not exist" Jul 22 04:46:26.281162 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.281140 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:46:26.282912 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:26.282886 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-865d8b7674-v7646"] Jul 22 04:46:27.430795 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:46:27.430724 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" path="/var/lib/kubelet/pods/f427e4ee-9121-4411-ac25-f09af5ecd5a1/volumes" Jul 22 04:47:27.083305 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083268 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083683 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9665b667-573a-49a2-b86c-cff04ac1b151" containerName="authorino" Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083694 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="9665b667-573a-49a2-b86c-cff04ac1b151" containerName="authorino" Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083708 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" containerName="manager" Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083714 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" containerName="manager" Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083785 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="9665b667-573a-49a2-b86c-cff04ac1b151" containerName="authorino" Jul 22 04:47:27.085609 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.083792 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f427e4ee-9121-4411-ac25-f09af5ecd5a1" containerName="manager" Jul 22 04:47:27.086632 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.086616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.089037 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.088999 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-chh6j\"" Jul 22 04:47:27.090249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.090227 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 04:47:27.090349 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.090230 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 04:47:27.096612 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.096587 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:27.162637 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.162615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.162739 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.162651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghrdk\" (UniqueName: \"kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.263126 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.263100 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.263222 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.263137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghrdk\" (UniqueName: \"kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.265482 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.265455 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.270894 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.270869 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghrdk\" (UniqueName: \"kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk\") pod \"maas-controller-865d8b7674-fhfzm\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.398126 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.398077 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:27.529898 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.529875 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:27.531926 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:47:27.531898 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c93171_b716_4f30_8f5c_b73b683f7a46.slice/crio-dca9a7d3abf4df309d4291f74c03eef0d79aee8a6da8aea40d8b0e21ce4352c7 WatchSource:0}: Error finding container dca9a7d3abf4df309d4291f74c03eef0d79aee8a6da8aea40d8b0e21ce4352c7: Status 404 returned error can't find the container with id dca9a7d3abf4df309d4291f74c03eef0d79aee8a6da8aea40d8b0e21ce4352c7 Jul 22 04:47:27.581184 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:27.581157 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-fhfzm" event={"ID":"d7c93171-b716-4f30-8f5c-b73b683f7a46","Type":"ContainerStarted","Data":"dca9a7d3abf4df309d4291f74c03eef0d79aee8a6da8aea40d8b0e21ce4352c7"} Jul 22 04:47:28.587946 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:28.587912 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-fhfzm" event={"ID":"d7c93171-b716-4f30-8f5c-b73b683f7a46","Type":"ContainerStarted","Data":"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b"} Jul 22 04:47:28.588364 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:28.587968 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:28.605282 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:28.605084 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865d8b7674-fhfzm" podStartSLOduration=1.129027489 podStartE2EDuration="1.605065733s" totalImagesPullingTime="476.038244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:47:28.604889453 +0000 UTC m=+863.653157717" watchObservedRunningTime="2026-07-22 04:47:28.605065733 +0000 UTC m=+863.653333993" Jul 22 04:47:39.595747 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:39.595705 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:44.231077 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.231040 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:44.231511 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.231229 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-865d8b7674-fhfzm" podUID="d7c93171-b716-4f30-8f5c-b73b683f7a46" containerName="manager" containerID="cri-o://f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b" gracePeriod=30 Jul 22 04:47:44.479240 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.479215 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:44.602309 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.602272 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghrdk\" (UniqueName: \"kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk\") pod \"d7c93171-b716-4f30-8f5c-b73b683f7a46\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " Jul 22 04:47:44.602457 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.602377 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert\") pod \"d7c93171-b716-4f30-8f5c-b73b683f7a46\" (UID: \"d7c93171-b716-4f30-8f5c-b73b683f7a46\") " Jul 22 04:47:44.604505 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.604475 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d7c93171-b716-4f30-8f5c-b73b683f7a46" (UID: "d7c93171-b716-4f30-8f5c-b73b683f7a46"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:47:44.604644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.604596 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk" (OuterVolumeSpecName: "kube-api-access-ghrdk") pod "d7c93171-b716-4f30-8f5c-b73b683f7a46" (UID: "d7c93171-b716-4f30-8f5c-b73b683f7a46"). InnerVolumeSpecName "kube-api-access-ghrdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:47:44.675371 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.675343 2555 generic.go:356] "Generic (PLEG): container finished" podID="d7c93171-b716-4f30-8f5c-b73b683f7a46" containerID="f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b" exitCode=0 Jul 22 04:47:44.675478 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.675389 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-fhfzm" Jul 22 04:47:44.675478 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.675425 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-fhfzm" event={"ID":"d7c93171-b716-4f30-8f5c-b73b683f7a46","Type":"ContainerDied","Data":"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b"} Jul 22 04:47:44.675554 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.675478 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-fhfzm" event={"ID":"d7c93171-b716-4f30-8f5c-b73b683f7a46","Type":"ContainerDied","Data":"dca9a7d3abf4df309d4291f74c03eef0d79aee8a6da8aea40d8b0e21ce4352c7"} Jul 22 04:47:44.675554 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.675501 2555 scope.go:117] "RemoveContainer" containerID="f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b" Jul 22 04:47:44.684502 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.684487 2555 scope.go:117] "RemoveContainer" containerID="f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b" Jul 22 04:47:44.684784 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:47:44.684766 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b\": container with ID starting with f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b not found: ID does not exist" containerID="f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b" Jul 22 04:47:44.684841 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.684792 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b"} err="failed to get container status \"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b\": rpc error: code = NotFound desc = could not find container \"f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b\": container with ID starting with f6ab8801f4609627033c4d589fac65a804d701ef9adf745f34886b4e0087758b not found: ID does not exist" Jul 22 04:47:44.697059 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.697038 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:44.698955 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.698933 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-865d8b7674-fhfzm"] Jul 22 04:47:44.703542 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.703526 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d7c93171-b716-4f30-8f5c-b73b683f7a46-webhook-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:47:44.703594 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:44.703547 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghrdk\" (UniqueName: \"kubernetes.io/projected/d7c93171-b716-4f30-8f5c-b73b683f7a46-kube-api-access-ghrdk\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:47:45.430924 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.430893 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c93171-b716-4f30-8f5c-b73b683f7a46" path="/var/lib/kubelet/pods/d7c93171-b716-4f30-8f5c-b73b683f7a46/volumes" Jul 22 04:47:45.467220 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.467196 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-865d8b7674-wczgw"] Jul 22 04:47:45.467600 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.467584 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c93171-b716-4f30-8f5c-b73b683f7a46" containerName="manager" Jul 22 04:47:45.467662 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.467602 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c93171-b716-4f30-8f5c-b73b683f7a46" containerName="manager" Jul 22 04:47:45.467731 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.467719 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c93171-b716-4f30-8f5c-b73b683f7a46" containerName="manager" Jul 22 04:47:45.472060 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.472042 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.475304 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.475269 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-chh6j\"" Jul 22 04:47:45.475304 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.475298 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 04:47:45.475569 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.475322 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 04:47:45.478339 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.478316 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-wczgw"] Jul 22 04:47:45.510373 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.510346 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00019a52-7e92-4e78-9628-788953f92677-webhook-cert\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.510497 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.510479 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s99g\" (UniqueName: \"kubernetes.io/projected/00019a52-7e92-4e78-9628-788953f92677-kube-api-access-4s99g\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.611743 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.611718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s99g\" (UniqueName: \"kubernetes.io/projected/00019a52-7e92-4e78-9628-788953f92677-kube-api-access-4s99g\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.611846 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.611775 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00019a52-7e92-4e78-9628-788953f92677-webhook-cert\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.614152 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.614134 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/00019a52-7e92-4e78-9628-788953f92677-webhook-cert\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.619958 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.619928 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s99g\" (UniqueName: \"kubernetes.io/projected/00019a52-7e92-4e78-9628-788953f92677-kube-api-access-4s99g\") pod \"maas-controller-865d8b7674-wczgw\" (UID: \"00019a52-7e92-4e78-9628-788953f92677\") " pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.783167 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.783144 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:45.910636 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:45.910607 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-865d8b7674-wczgw"] Jul 22 04:47:45.913879 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:47:45.913852 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00019a52_7e92_4e78_9628_788953f92677.slice/crio-90d91b52ebe4bca8260a1f3f59f41cb2aad77ca13bd23252134c4db848671b7f WatchSource:0}: Error finding container 90d91b52ebe4bca8260a1f3f59f41cb2aad77ca13bd23252134c4db848671b7f: Status 404 returned error can't find the container with id 90d91b52ebe4bca8260a1f3f59f41cb2aad77ca13bd23252134c4db848671b7f Jul 22 04:47:46.688767 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:46.688732 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-wczgw" event={"ID":"00019a52-7e92-4e78-9628-788953f92677","Type":"ContainerStarted","Data":"9670d568aa1d8d62db126750f65bbc863e8cb84cadb6a3d34a89976dc78d206e"} Jul 22 04:47:46.689200 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:46.688787 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-865d8b7674-wczgw" event={"ID":"00019a52-7e92-4e78-9628-788953f92677","Type":"ContainerStarted","Data":"90d91b52ebe4bca8260a1f3f59f41cb2aad77ca13bd23252134c4db848671b7f"} Jul 22 04:47:46.689200 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:46.688845 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:47:46.705400 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:46.705354 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-865d8b7674-wczgw" podStartSLOduration=1.408392448 podStartE2EDuration="1.70534105s" totalImagesPullingTime="296.948602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:47:46.703214539 +0000 UTC m=+881.751482797" watchObservedRunningTime="2026-07-22 04:47:46.70534105 +0000 UTC m=+881.753609309" Jul 22 04:47:57.697149 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:47:57.697047 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-865d8b7674-wczgw" Jul 22 04:48:05.383313 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:48:05.383286 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:48:05.383767 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:48:05.383472 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:52:21.349380 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:21.349342 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:52:21.349972 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:21.349750 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" podUID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" containerName="manager" containerID="cri-o://ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca" gracePeriod=10 Jul 22 04:52:25.007509 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.007485 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:52:25.156158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.156072 2555 generic.go:356] "Generic (PLEG): container finished" podID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" containerID="ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca" exitCode=0 Jul 22 04:52:25.156158 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.156108 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" Jul 22 04:52:25.156392 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.156152 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" event={"ID":"3bc7d412-0c28-41af-8fc5-63ec4a470e7d","Type":"ContainerDied","Data":"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca"} Jul 22 04:52:25.156392 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.156208 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz" event={"ID":"3bc7d412-0c28-41af-8fc5-63ec4a470e7d","Type":"ContainerDied","Data":"88505ebe31bcc7f65c3e22993efd0e495601b6780df0ccccea336e0ac97ea45c"} Jul 22 04:52:25.156392 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.156244 2555 scope.go:117] "RemoveContainer" containerID="ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca" Jul 22 04:52:25.165741 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.165725 2555 scope.go:117] "RemoveContainer" containerID="ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca" Jul 22 04:52:25.166003 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:52:25.165983 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca\": container with ID starting with ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca not found: ID does not exist" containerID="ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca" Jul 22 04:52:25.166073 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.166011 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca"} err="failed to get container status \"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca\": rpc error: code = NotFound desc = could not find container \"ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca\": container with ID starting with ea05151604cc326e5082b4728dda8e889642b992c31ca5b925f37945575b43ca not found: ID does not exist" Jul 22 04:52:25.193560 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.193540 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume\") pod \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " Jul 22 04:52:25.193670 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.193652 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfbmq\" (UniqueName: \"kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq\") pod \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\" (UID: \"3bc7d412-0c28-41af-8fc5-63ec4a470e7d\") " Jul 22 04:52:25.193857 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.193838 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3bc7d412-0c28-41af-8fc5-63ec4a470e7d" (UID: "3bc7d412-0c28-41af-8fc5-63ec4a470e7d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:52:25.195597 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.195571 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq" (OuterVolumeSpecName: "kube-api-access-vfbmq") pod "3bc7d412-0c28-41af-8fc5-63ec4a470e7d" (UID: "3bc7d412-0c28-41af-8fc5-63ec4a470e7d"). InnerVolumeSpecName "kube-api-access-vfbmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:52:25.294539 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.294510 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfbmq\" (UniqueName: \"kubernetes.io/projected/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-kube-api-access-vfbmq\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:52:25.294539 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.294534 2555 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3bc7d412-0c28-41af-8fc5-63ec4a470e7d-extensions-socket-volume\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:52:25.471541 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.471510 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:52:25.475272 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:25.475251 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bl8qz"] Jul 22 04:52:27.435307 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:52:27.433413 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" path="/var/lib/kubelet/pods/3bc7d412-0c28-41af-8fc5-63ec4a470e7d/volumes" Jul 22 04:53:05.420157 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:05.420127 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:53:05.421779 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:05.421750 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:53:22.499840 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.499800 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt"] Jul 22 04:53:22.500397 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.500270 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" containerName="manager" Jul 22 04:53:22.500397 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.500283 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" containerName="manager" Jul 22 04:53:22.500397 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.500355 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc7d412-0c28-41af-8fc5-63ec4a470e7d" containerName="manager" Jul 22 04:53:22.504769 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.504749 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.507284 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.507258 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v4dp9\"" Jul 22 04:53:22.515209 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.515179 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt"] Jul 22 04:53:22.657225 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.657192 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a106dee-2e29-4910-977d-e4288c6502f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.657373 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.657245 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86sd8\" (UniqueName: \"kubernetes.io/projected/6a106dee-2e29-4910-977d-e4288c6502f1-kube-api-access-86sd8\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.757996 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.757911 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86sd8\" (UniqueName: \"kubernetes.io/projected/6a106dee-2e29-4910-977d-e4288c6502f1-kube-api-access-86sd8\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.758139 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.758007 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a106dee-2e29-4910-977d-e4288c6502f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.758342 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.758320 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a106dee-2e29-4910-977d-e4288c6502f1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.771469 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.771441 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86sd8\" (UniqueName: \"kubernetes.io/projected/6a106dee-2e29-4910-977d-e4288c6502f1-kube-api-access-86sd8\") pod \"kuadrant-operator-controller-manager-86878c769f-7gdpt\" (UID: \"6a106dee-2e29-4910-977d-e4288c6502f1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.816139 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.816108 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:22.955533 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:22.955484 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt"] Jul 22 04:53:22.958035 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:53:22.957996 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a106dee_2e29_4910_977d_e4288c6502f1.slice/crio-f74c2b872511cfd18cf4d2869493c06d0b5d54ca40191b0a393cee02b6515da6 WatchSource:0}: Error finding container f74c2b872511cfd18cf4d2869493c06d0b5d54ca40191b0a393cee02b6515da6: Status 404 returned error can't find the container with id f74c2b872511cfd18cf4d2869493c06d0b5d54ca40191b0a393cee02b6515da6 Jul 22 04:53:23.474434 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:23.474352 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" event={"ID":"6a106dee-2e29-4910-977d-e4288c6502f1","Type":"ContainerStarted","Data":"ca1caab099bcab3b09762aa8495b300277b6461ef9ef00ea1a08e8a96925bbad"} Jul 22 04:53:23.474580 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:23.474438 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" event={"ID":"6a106dee-2e29-4910-977d-e4288c6502f1","Type":"ContainerStarted","Data":"f74c2b872511cfd18cf4d2869493c06d0b5d54ca40191b0a393cee02b6515da6"} Jul 22 04:53:23.474580 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:23.474468 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:53:23.497527 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:23.497485 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" podStartSLOduration=1.497471363 podStartE2EDuration="1.497471363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:53:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:53:23.496330751 +0000 UTC m=+1218.544599012" watchObservedRunningTime="2026-07-22 04:53:23.497471363 +0000 UTC m=+1218.545739622" Jul 22 04:53:34.481641 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:53:34.481594 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7gdpt" Jul 22 04:56:12.566422 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.566385 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:56:12.570177 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.570155 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.572664 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.572649 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-cf20ba46-kserve-self-signed-certs\"" Jul 22 04:56:12.585666 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.585633 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:56:12.619790 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.619886 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619810 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.619886 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619834 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.619973 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619904 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwwwr\" (UniqueName: \"kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.619973 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.620085 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.619992 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721105 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721073 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721246 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721136 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721246 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721203 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721246 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721238 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721468 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwwwr\" (UniqueName: \"kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721468 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721578 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721531 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721638 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721599 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.721789 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.721770 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.723536 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.723511 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.723805 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.723787 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.728524 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.728501 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwwwr\" (UniqueName: \"kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr\") pod \"e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:12.880729 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:12.880657 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:13.016430 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:13.016404 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:56:13.018604 ip-10-0-133-75 kubenswrapper[2555]: W0722 04:56:13.018579 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod059cd767_8bc5_4a4d_97a8_7c8da3ac27c5.slice/crio-457b501cc90e858f5975cf8c778dcd094f794bbf369356ad3c0f67c627577501 WatchSource:0}: Error finding container 457b501cc90e858f5975cf8c778dcd094f794bbf369356ad3c0f67c627577501: Status 404 returned error can't find the container with id 457b501cc90e858f5975cf8c778dcd094f794bbf369356ad3c0f67c627577501 Jul 22 04:56:13.342784 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:13.342748 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" event={"ID":"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5","Type":"ContainerStarted","Data":"eb2f6a84a516fbfc179ca661343a57d63fd8607c3e33058458e4adc5ffd5cede"} Jul 22 04:56:13.342927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:13.342795 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" event={"ID":"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5","Type":"ContainerStarted","Data":"457b501cc90e858f5975cf8c778dcd094f794bbf369356ad3c0f67c627577501"} Jul 22 04:56:13.358884 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:13.358835 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" podStartSLOduration=1.358819424 podStartE2EDuration="1.358819424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:56:13.357599303 +0000 UTC m=+1388.405867562" watchObservedRunningTime="2026-07-22 04:56:13.358819424 +0000 UTC m=+1388.407087681" Jul 22 04:56:22.880983 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:22.880901 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:22.880983 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:22.880955 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:22.892632 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:22.892611 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:23.410650 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:23.410626 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:56:24.260733 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.260698 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:56:24.265000 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.264976 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.267299 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.267278 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-cf20ba46-kserve-self-signed-certs\"" Jul 22 04:56:24.272556 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.272532 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:56:24.414745 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414715 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhc8p\" (UniqueName: \"kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.414875 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414773 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.414875 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414795 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.414875 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414854 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.414995 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414887 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.414995 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.414915 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.515660 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhc8p\" (UniqueName: \"kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.515803 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515669 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.515803 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515695 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.515923 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515840 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.515923 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515888 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.516059 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.515939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.516137 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.516115 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.516281 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.516260 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.516342 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.516269 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.517999 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.517976 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.518272 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.518256 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.523855 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.523826 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhc8p\" (UniqueName: \"kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p\") pod \"e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.575717 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.575694 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:24.702644 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:24.702619 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:56:25.412771 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:25.412736 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" event={"ID":"e137fdd8-622f-47a1-85a6-4e74ef9f5901","Type":"ContainerStarted","Data":"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb"} Jul 22 04:56:25.413165 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:25.412780 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" event={"ID":"e137fdd8-622f-47a1-85a6-4e74ef9f5901","Type":"ContainerStarted","Data":"9af30a97aa63a89f8d5bb754b42acf8261ed9a18720de66ddf7fc28683498329"} Jul 22 04:56:25.430516 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:25.430462 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" podStartSLOduration=1.430446083 podStartE2EDuration="1.430446083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:56:25.428752348 +0000 UTC m=+1400.477020606" watchObservedRunningTime="2026-07-22 04:56:25.430446083 +0000 UTC m=+1400.478714340" Jul 22 04:56:34.576403 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:34.576370 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:34.576936 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:34.576421 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:34.588337 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:34.588310 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:35.472741 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:35.472713 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:56:58.455633 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:58.455549 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:56:59.153017 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:56:59.152980 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:57:13.476103 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.476045 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" podUID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" containerName="main" containerID="cri-o://eb2f6a84a516fbfc179ca661343a57d63fd8607c3e33058458e4adc5ffd5cede" gracePeriod=45 Jul 22 04:57:13.654375 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.654342 2555 generic.go:356] "Generic (PLEG): container finished" podID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" containerID="eb2f6a84a516fbfc179ca661343a57d63fd8607c3e33058458e4adc5ffd5cede" exitCode=0 Jul 22 04:57:13.654519 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.654370 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" event={"ID":"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5","Type":"ContainerDied","Data":"eb2f6a84a516fbfc179ca661343a57d63fd8607c3e33058458e4adc5ffd5cede"} Jul 22 04:57:13.732614 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.732564 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:57:13.832738 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832712 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.832738 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832739 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.832902 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832785 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.832902 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832825 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.832902 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832868 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwwwr\" (UniqueName: \"kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.832902 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832898 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs\") pod \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\" (UID: \"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5\") " Jul 22 04:57:13.833150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.832928 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:13.833150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.833055 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache" (OuterVolumeSpecName: "model-cache") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:13.833150 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.833064 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home" (OuterVolumeSpecName: "home") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:13.833345 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.833326 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tmp-dir\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:13.833390 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.833347 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-home\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:13.833390 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.833356 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-model-cache\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:13.834886 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.834866 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:57:13.835343 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.835318 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm" (OuterVolumeSpecName: "dshm") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:13.835415 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.835337 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr" (OuterVolumeSpecName: "kube-api-access-cwwwr") pod "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" (UID: "059cd767-8bc5-4a4d-97a8-7c8da3ac27c5"). InnerVolumeSpecName "kube-api-access-cwwwr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:57:13.934249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.934225 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-dshm\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:13.934249 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.934248 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwwwr\" (UniqueName: \"kubernetes.io/projected/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-kube-api-access-cwwwr\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:13.934381 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:13.934259 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5-tls-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.173105 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.173047 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" podUID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" containerName="main" containerID="cri-o://cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb" gracePeriod=45 Jul 22 04:57:14.407920 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.407893 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:57:14.539934 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.539900 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.539949 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhc8p\" (UniqueName: \"kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.539965 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.539989 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540042 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540082 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm\") pod \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\" (UID: \"e137fdd8-622f-47a1-85a6-4e74ef9f5901\") " Jul 22 04:57:14.540275 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540239 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home" (OuterVolumeSpecName: "home") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:14.540552 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540310 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:14.540552 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540316 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache" (OuterVolumeSpecName: "model-cache") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:14.540552 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540422 2555 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-home\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.540552 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540435 2555 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-model-cache\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.540552 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.540449 2555 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tmp-dir\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.541935 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.541912 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:57:14.542061 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.541957 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p" (OuterVolumeSpecName: "kube-api-access-xhc8p") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "kube-api-access-xhc8p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:57:14.542061 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.541970 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm" (OuterVolumeSpecName: "dshm") pod "e137fdd8-622f-47a1-85a6-4e74ef9f5901" (UID: "e137fdd8-622f-47a1-85a6-4e74ef9f5901"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:57:14.641391 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.641365 2555 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e137fdd8-622f-47a1-85a6-4e74ef9f5901-tls-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.641391 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.641389 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhc8p\" (UniqueName: \"kubernetes.io/projected/e137fdd8-622f-47a1-85a6-4e74ef9f5901-kube-api-access-xhc8p\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.641518 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.641398 2555 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e137fdd8-622f-47a1-85a6-4e74ef9f5901-dshm\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 04:57:14.661090 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.661066 2555 generic.go:356] "Generic (PLEG): container finished" podID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" containerID="cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb" exitCode=0 Jul 22 04:57:14.661185 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.661107 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" Jul 22 04:57:14.661185 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.661144 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" event={"ID":"e137fdd8-622f-47a1-85a6-4e74ef9f5901","Type":"ContainerDied","Data":"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb"} Jul 22 04:57:14.661269 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.661184 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd" event={"ID":"e137fdd8-622f-47a1-85a6-4e74ef9f5901","Type":"ContainerDied","Data":"9af30a97aa63a89f8d5bb754b42acf8261ed9a18720de66ddf7fc28683498329"} Jul 22 04:57:14.661269 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.661203 2555 scope.go:117] "RemoveContainer" containerID="cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb" Jul 22 04:57:14.662514 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.662398 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" event={"ID":"059cd767-8bc5-4a4d-97a8-7c8da3ac27c5","Type":"ContainerDied","Data":"457b501cc90e858f5975cf8c778dcd094f794bbf369356ad3c0f67c627577501"} Jul 22 04:57:14.662514 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.662410 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd" Jul 22 04:57:14.670927 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.670913 2555 scope.go:117] "RemoveContainer" containerID="cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb" Jul 22 04:57:14.671193 ip-10-0-133-75 kubenswrapper[2555]: E0722 04:57:14.671173 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb\": container with ID starting with cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb not found: ID does not exist" containerID="cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb" Jul 22 04:57:14.671277 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.671203 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb"} err="failed to get container status \"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb\": rpc error: code = NotFound desc = could not find container \"cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb\": container with ID starting with cad12a4dd251ff40997eb51dbdc27bdd4bffe4ad1e13accc6d09bef4207f34bb not found: ID does not exist" Jul 22 04:57:14.671277 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.671226 2555 scope.go:117] "RemoveContainer" containerID="eb2f6a84a516fbfc179ca661343a57d63fd8607c3e33058458e4adc5ffd5cede" Jul 22 04:57:14.682677 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.682655 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:57:14.686434 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.686415 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-cf20ba46-kserve-5c888787d7-q74kd"] Jul 22 04:57:14.695908 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.695889 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:57:14.702587 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:14.702568 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-cf20ba46-kserve-689d5dcff-hmfsd"] Jul 22 04:57:15.430450 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:15.430413 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" path="/var/lib/kubelet/pods/059cd767-8bc5-4a4d-97a8-7c8da3ac27c5/volumes" Jul 22 04:57:15.430864 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:57:15.430843 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" path="/var/lib/kubelet/pods/e137fdd8-622f-47a1-85a6-4e74ef9f5901/volumes" Jul 22 04:58:05.449725 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:58:05.449698 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 04:58:05.454095 ip-10-0-133-75 kubenswrapper[2555]: I0722 04:58:05.454078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:00:38.349625 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.349590 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:00:38.350264 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350116 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" containerName="main" Jul 22 05:00:38.350264 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350143 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" containerName="main" Jul 22 05:00:38.350264 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350167 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" containerName="main" Jul 22 05:00:38.350264 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350176 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" containerName="main" Jul 22 05:00:38.350517 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350303 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="059cd767-8bc5-4a4d-97a8-7c8da3ac27c5" containerName="main" Jul 22 05:00:38.350517 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.350316 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="e137fdd8-622f-47a1-85a6-4e74ef9f5901" containerName="main" Jul 22 05:00:38.353102 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.353083 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.355622 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.355602 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9e76c960-openshift-default-dockercfg-zl7rm\"" Jul 22 05:00:38.363310 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.363290 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:00:38.437006 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.436981 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d66v\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437125 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437014 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437125 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437072 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437125 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437093 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437154 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437172 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437220 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.437255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.437235 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538558 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538526 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d66v\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538558 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538559 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538585 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538603 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538651 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538668 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538687 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.538776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538733 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539126 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.538777 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539234 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.539204 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539301 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.539238 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539301 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.539278 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539411 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.539323 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.539411 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.539377 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.540950 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.540927 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.541343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.541324 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.546461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.546436 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.546983 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.546961 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d66v\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v\") pod \"e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.664264 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.664206 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:38.804435 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.804401 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:00:38.806310 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:00:38.806270 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59f1ef0_13ba_4b5a_a08b_c4cf77e50a5a.slice/crio-eecbb6cfc9d21c523e4275fbecc9abe1d5a77bf098fb3cd2a4df634a8fe4f9ce WatchSource:0}: Error finding container eecbb6cfc9d21c523e4275fbecc9abe1d5a77bf098fb3cd2a4df634a8fe4f9ce: Status 404 returned error can't find the container with id eecbb6cfc9d21c523e4275fbecc9abe1d5a77bf098fb3cd2a4df634a8fe4f9ce Jul 22 05:00:38.808799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.808760 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:38.808946 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.808843 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:38.808946 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:38.808893 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:39.715610 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:39.715576 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerStarted","Data":"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc"} Jul 22 05:00:39.715958 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:39.715627 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerStarted","Data":"eecbb6cfc9d21c523e4275fbecc9abe1d5a77bf098fb3cd2a4df634a8fe4f9ce"} Jul 22 05:00:39.735286 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:39.735243 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podStartSLOduration=1.735229525 podStartE2EDuration="1.735229525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:00:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:00:39.734144016 +0000 UTC m=+1654.782412262" watchObservedRunningTime="2026-07-22 05:00:39.735229525 +0000 UTC m=+1654.783497784" Jul 22 05:00:40.664635 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:40.664598 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:40.669501 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:40.669476 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:40.721934 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:40.721905 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:40.722715 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:40.722691 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:56.764174 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:56.764134 2555 patch_prober.go:28] interesting pod/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 05:00:56.765427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:56.764204 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 05:00:57.815504 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.815468 2555 generic.go:356] "Generic (PLEG): container finished" podID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerID="b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc" exitCode=0 Jul 22 05:00:57.815867 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.815541 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerDied","Data":"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc"} Jul 22 05:00:57.815922 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.815908 2555 scope.go:117] "RemoveContainer" containerID="b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc" Jul 22 05:00:57.816916 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.816887 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:57.816990 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.816972 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:57.817075 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:57.817056 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:00:58.825728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:58.825688 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerStarted","Data":"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3"} Jul 22 05:00:59.665235 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:59.665195 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:00:59.665403 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:59.665369 2555 patch_prober.go:28] interesting pod/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 05:00:59.665448 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:00:59.665426 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 05:01:00.664880 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:00.664843 2555 patch_prober.go:28] interesting pod/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 05:01:00.665283 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:00.664900 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 05:01:01.665494 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:01.665463 2555 patch_prober.go:28] interesting pod/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 05:01:01.665896 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:01.665522 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 05:01:02.665467 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:02.665433 2555 patch_prober.go:28] interesting pod/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 22 05:01:02.665613 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:02.665500 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 22 05:01:03.668444 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:03.668412 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:01:03.668858 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:03.668700 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:01:03.669517 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:03.669492 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:01:21.880702 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:21.880666 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:01:21.881277 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:21.880881 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" containerID="cri-o://deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3" gracePeriod=30 Jul 22 05:01:22.890538 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.890504 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:01:22.893475 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.893451 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.895879 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.895858 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-dec153cb-gw-openshift-default-dockercfg-tclsz\"" Jul 22 05:01:22.906405 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.906375 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:01:22.913255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913229 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913370 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913263 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913370 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913293 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913370 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913339 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913545 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913375 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8bft\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913545 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913545 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913417 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913545 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:22.913545 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:22.913514 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014055 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014004 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014071 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014092 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8bft\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014114 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014156 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014198 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014569 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014409 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014569 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014541 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014237 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014690 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014704 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014806 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.014953 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.014927 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.015289 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.015273 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.016989 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.016962 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.017215 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.017197 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.021682 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.021662 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.021803 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.021715 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8bft\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft\") pod \"e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.203984 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.203917 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:23.351924 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.351346 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:01:23.352076 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:01:23.351988 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfebfa47_32a4_42ab_a94b_2aa202202456.slice/crio-f2a3170145964618d06fa92572c9ae34e8b429ac220dc89a38be2530220c2083 WatchSource:0}: Error finding container f2a3170145964618d06fa92572c9ae34e8b429ac220dc89a38be2530220c2083: Status 404 returned error can't find the container with id f2a3170145964618d06fa92572c9ae34e8b429ac220dc89a38be2530220c2083 Jul 22 05:01:23.355301 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.355266 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:23.355395 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.355330 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:23.355395 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.355358 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:23.956486 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.956447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerStarted","Data":"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1"} Jul 22 05:01:23.956961 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.956508 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerStarted","Data":"f2a3170145964618d06fa92572c9ae34e8b429ac220dc89a38be2530220c2083"} Jul 22 05:01:23.976728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:23.976678 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podStartSLOduration=1.976662911 podStartE2EDuration="1.976662911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:01:23.975071201 +0000 UTC m=+1699.023339460" watchObservedRunningTime="2026-07-22 05:01:23.976662911 +0000 UTC m=+1699.024931170" Jul 22 05:01:24.204918 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:24.204885 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:24.209913 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:24.209850 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:24.962063 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:24.962028 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:24.962817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:24.962797 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:27.137627 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.137603 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:01:27.152810 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152742 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.152810 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152807 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152826 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152846 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d66v\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152872 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152961 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152996 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153292 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.152994 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:01:27.153292 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153057 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153292 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153091 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo\") pod \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\" (UID: \"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a\") " Jul 22 05:01:27.153441 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153345 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:01:27.153511 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153478 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:01:27.153600 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153579 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.153679 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153608 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.153733 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153684 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:01:27.153882 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.153834 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:01:27.155099 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.155002 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:01:27.155858 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.155619 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:01:27.156244 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.156216 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:01:27.158577 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.158551 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v" (OuterVolumeSpecName: "kube-api-access-9d66v") pod "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" (UID: "a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a"). InnerVolumeSpecName "kube-api-access-9d66v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:01:27.254667 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254643 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254667 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254666 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254795 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254678 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254795 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254685 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254795 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254693 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254795 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254700 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d66v\" (UniqueName: \"kubernetes.io/projected/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-kube-api-access-9d66v\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.254795 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.254708 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:01:27.978613 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.978569 2555 generic.go:356] "Generic (PLEG): container finished" podID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerID="deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3" exitCode=0 Jul 22 05:01:27.978777 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.978646 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" Jul 22 05:01:27.978777 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.978638 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerDied","Data":"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3"} Jul 22 05:01:27.978855 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.978818 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg" event={"ID":"a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a","Type":"ContainerDied","Data":"eecbb6cfc9d21c523e4275fbecc9abe1d5a77bf098fb3cd2a4df634a8fe4f9ce"} Jul 22 05:01:27.978890 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.978856 2555 scope.go:117] "RemoveContainer" containerID="deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3" Jul 22 05:01:27.987904 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.987888 2555 scope.go:117] "RemoveContainer" containerID="b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc" Jul 22 05:01:27.996443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.996294 2555 scope.go:117] "RemoveContainer" containerID="deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3" Jul 22 05:01:27.996443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.996381 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:01:27.996574 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:01:27.996557 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3\": container with ID starting with deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3 not found: ID does not exist" containerID="deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3" Jul 22 05:01:27.996618 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.996590 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3"} err="failed to get container status \"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3\": rpc error: code = NotFound desc = could not find container \"deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3\": container with ID starting with deafaebfbaa981d9d7be22cceb3941191f6d3d2a0930ad1f747eed9ea42515a3 not found: ID does not exist" Jul 22 05:01:27.996618 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.996607 2555 scope.go:117] "RemoveContainer" containerID="b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc" Jul 22 05:01:27.996879 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:01:27.996863 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc\": container with ID starting with b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc not found: ID does not exist" containerID="b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc" Jul 22 05:01:27.996935 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:27.996882 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc"} err="failed to get container status \"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc\": rpc error: code = NotFound desc = could not find container \"b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc\": container with ID starting with b38a80e7899a77583f029be74335641e4c2bfc3184a2f80e42d7044e0b993fcc not found: ID does not exist" Jul 22 05:01:28.001558 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:28.001538 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9e76c960-openshift-default-5c886584b9-pwwrg"] Jul 22 05:01:29.434731 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:29.434696 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" path="/var/lib/kubelet/pods/a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a/volumes" Jul 22 05:01:36.022552 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.022524 2555 generic.go:356] "Generic (PLEG): container finished" podID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerID="f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1" exitCode=0 Jul 22 05:01:36.022905 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.022557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerDied","Data":"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1"} Jul 22 05:01:36.022972 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.022958 2555 scope.go:117] "RemoveContainer" containerID="f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1" Jul 22 05:01:36.024012 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.023984 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:36.024121 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.024103 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:36.024185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:36.024165 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:01:37.033283 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:37.033249 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerStarted","Data":"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e"} Jul 22 05:01:37.204692 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:37.204599 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:37.205943 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:37.205910 2555 patch_prober.go:28] interesting pod/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 22 05:01:37.206075 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:37.205989 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 22 05:01:38.205585 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:38.205124 2555 patch_prober.go:28] interesting pod/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 22 05:01:38.205585 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:38.205195 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 22 05:01:39.204894 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:39.204847 2555 patch_prober.go:28] interesting pod/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 22 05:01:39.205114 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:39.204916 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 22 05:01:39.962324 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:39.962275 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:40.204960 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:40.204911 2555 patch_prober.go:28] interesting pod/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 22 05:01:40.205198 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:40.204991 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 22 05:01:41.209075 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:41.209042 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:01:41.210004 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:01:41.209982 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:02:05.709584 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:05.709537 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:02:05.710173 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:05.709800 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" containerID="cri-o://7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e" gracePeriod=30 Jul 22 05:02:06.488469 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.488435 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:06.489135 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489118 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.489194 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489140 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.489194 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489182 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.489194 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489192 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.489327 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489315 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.489367 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.489334 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a59f1ef0-13ba-4b5a-a08b-c4cf77e50a5a" containerName="istio-proxy" Jul 22 05:02:06.494090 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.494068 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.496403 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.496382 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-303fe40d-openshift-default-dockercfg-pb88k\"" Jul 22 05:02:06.504895 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.504867 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:06.603320 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603269 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bng7n\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603402 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603480 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603674 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603736 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603674 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603736 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603702 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603841 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603751 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.603841 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.603820 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705039 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.704942 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705039 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.704981 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705039 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705046 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bng7n\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705135 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705169 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705270 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705657 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705414 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705657 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705410 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705772 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705651 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.705829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.705802 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.706367 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.706337 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.707951 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.707917 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.708337 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.708315 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.712838 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.712807 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.713253 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.713230 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bng7n\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n\") pod \"e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.806349 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.806320 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:06.941886 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.941848 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:06.943571 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:02:06.943544 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602994c1_640e_46ba_a9c5_aa49be6cf943.slice/crio-63818b00e8176b65678aa9bb0dffc553abca13578fe72b807e3503e2e401fd5f WatchSource:0}: Error finding container 63818b00e8176b65678aa9bb0dffc553abca13578fe72b807e3503e2e401fd5f: Status 404 returned error can't find the container with id 63818b00e8176b65678aa9bb0dffc553abca13578fe72b807e3503e2e401fd5f Jul 22 05:02:06.945735 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.945704 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:06.945808 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.945763 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:06.945808 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:06.945793 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:07.193104 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:07.193069 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerStarted","Data":"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4"} Jul 22 05:02:07.193251 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:07.193136 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerStarted","Data":"63818b00e8176b65678aa9bb0dffc553abca13578fe72b807e3503e2e401fd5f"} Jul 22 05:02:07.211201 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:07.211107 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podStartSLOduration=1.211086476 podStartE2EDuration="1.211086476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:02:07.210074619 +0000 UTC m=+1742.258342877" watchObservedRunningTime="2026-07-22 05:02:07.211086476 +0000 UTC m=+1742.259354736" Jul 22 05:02:07.806581 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:07.806539 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:07.811190 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:07.811163 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:08.199909 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:08.199810 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:08.200730 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:08.200706 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:10.965171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:10.965147 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:02:11.044430 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044402 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044436 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8bft\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044462 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044504 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044544 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044760 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044700 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044767 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044779 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:11.044817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044811 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044963 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044841 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data\") pod \"bfebfa47-32a4-42ab-a94b-2aa202202456\" (UID: \"bfebfa47-32a4-42ab-a94b-2aa202202456\") " Jul 22 05:02:11.044963 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.044930 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:11.045128 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.045103 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:11.045312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.045261 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:02:11.045312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.045233 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.045427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.045325 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.045645 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.045624 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data" (OuterVolumeSpecName: "istio-data") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:11.047177 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.047150 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft" (OuterVolumeSpecName: "kube-api-access-m8bft") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "kube-api-access-m8bft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:02:11.047177 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.047166 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:02:11.047311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.047183 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token" (OuterVolumeSpecName: "istio-token") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:02:11.047311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.047166 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bfebfa47-32a4-42ab-a94b-2aa202202456" (UID: "bfebfa47-32a4-42ab-a94b-2aa202202456"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:11.145671 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145649 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145671 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145670 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145678 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145686 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8bft\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-kube-api-access-m8bft\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145694 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145704 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfebfa47-32a4-42ab-a94b-2aa202202456-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.145799 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.145712 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfebfa47-32a4-42ab-a94b-2aa202202456-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:11.218312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.218287 2555 generic.go:356] "Generic (PLEG): container finished" podID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerID="7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e" exitCode=0 Jul 22 05:02:11.218400 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.218342 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" Jul 22 05:02:11.218400 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.218372 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerDied","Data":"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e"} Jul 22 05:02:11.218473 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.218422 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf" event={"ID":"bfebfa47-32a4-42ab-a94b-2aa202202456","Type":"ContainerDied","Data":"f2a3170145964618d06fa92572c9ae34e8b429ac220dc89a38be2530220c2083"} Jul 22 05:02:11.218473 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.218444 2555 scope.go:117] "RemoveContainer" containerID="7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e" Jul 22 05:02:11.227936 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.227913 2555 scope.go:117] "RemoveContainer" containerID="f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1" Jul 22 05:02:11.235526 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.235508 2555 scope.go:117] "RemoveContainer" containerID="7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e" Jul 22 05:02:11.235754 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:02:11.235738 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e\": container with ID starting with 7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e not found: ID does not exist" containerID="7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e" Jul 22 05:02:11.235794 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.235763 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e"} err="failed to get container status \"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e\": rpc error: code = NotFound desc = could not find container \"7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e\": container with ID starting with 7c2d1acb0574f59f83abc2d1c89e29e3cffcf3964c10253f270ea4ce5387228e not found: ID does not exist" Jul 22 05:02:11.235794 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.235778 2555 scope.go:117] "RemoveContainer" containerID="f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1" Jul 22 05:02:11.236007 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:02:11.235993 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1\": container with ID starting with f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1 not found: ID does not exist" containerID="f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1" Jul 22 05:02:11.236055 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.236011 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1"} err="failed to get container status \"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1\": rpc error: code = NotFound desc = could not find container \"f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1\": container with ID starting with f0f580fe24db643ce107ad00723dadc93a13baee05860e2c3e6b95bd6d8e06a1 not found: ID does not exist" Jul 22 05:02:11.239844 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.239827 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:02:11.241709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.241690 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-dec153cb-gw-openshift-default-56b6cd9859-rstbf"] Jul 22 05:02:11.431042 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:11.430971 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" path="/var/lib/kubelet/pods/bfebfa47-32a4-42ab-a94b-2aa202202456/volumes" Jul 22 05:02:24.077554 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.076321 2555 patch_prober.go:28] interesting pod/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:38804->10.132.0.57:15021: read: connection reset by peer" start-of-body= Jul 22 05:02:24.077554 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.076386 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": read tcp 10.132.0.2:38804->10.132.0.57:15021: read: connection reset by peer" Jul 22 05:02:24.297982 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.297925 2555 generic.go:356] "Generic (PLEG): container finished" podID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerID="e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4" exitCode=0 Jul 22 05:02:24.297982 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.297966 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerDied","Data":"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4"} Jul 22 05:02:24.298359 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.298346 2555 scope.go:117] "RemoveContainer" containerID="e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4" Jul 22 05:02:24.299339 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.299308 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:24.299434 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.299392 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:24.299496 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:24.299445 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:25.313425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:25.313380 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerStarted","Data":"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02"} Jul 22 05:02:25.807432 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:25.807388 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:25.807643 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:25.807520 2555 patch_prober.go:28] interesting pod/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 22 05:02:25.807643 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:25.807578 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 22 05:02:26.807042 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:26.806988 2555 patch_prober.go:28] interesting pod/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 22 05:02:26.807515 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:26.807081 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 22 05:02:27.806950 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:27.806915 2555 patch_prober.go:28] interesting pod/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 22 05:02:27.807275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:27.807236 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 22 05:02:28.806872 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:28.806833 2555 patch_prober.go:28] interesting pod/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 22 05:02:28.807196 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:28.807165 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 22 05:02:29.810275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:29.810247 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:29.810733 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:29.810526 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:29.811382 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:29.811365 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:50.590779 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:50.590743 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:50.591311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:50.591035 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" containerID="cri-o://4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02" gracePeriod=30 Jul 22 05:02:51.329806 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.329774 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:02:51.330504 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.330481 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:02:51.330504 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.330502 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:02:51.330666 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.330577 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:02:51.333857 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.333838 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.335991 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.335971 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9e622b09-openshift-default-dockercfg-75wmm\"" Jul 22 05:02:51.344995 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.344973 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:02:51.513170 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513132 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513335 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513191 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513381 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513419 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513399 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513459 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513440 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513496 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513483 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513575 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513662 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513640 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhdtb\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.513750 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.513734 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615067 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.614983 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615067 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615047 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615098 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615131 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615205 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615289 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhdtb\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615334 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615351 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615534 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615446 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615992 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615834 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.615992 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.615879 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.616126 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.616095 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.617419 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.617390 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.617683 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.617666 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.622278 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.622258 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.622402 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.622382 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhdtb\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb\") pod \"e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.644779 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.644758 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:51.783686 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.783662 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:02:51.785902 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:02:51.785874 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f43116_5068_4d26_af12_042f0f726ae3.slice/crio-b18ccf9c89477034d55af3adc110f007ea58d37ea310dd1a7c564065ec746f65 WatchSource:0}: Error finding container b18ccf9c89477034d55af3adc110f007ea58d37ea310dd1a7c564065ec746f65: Status 404 returned error can't find the container with id b18ccf9c89477034d55af3adc110f007ea58d37ea310dd1a7c564065ec746f65 Jul 22 05:02:51.788102 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.788075 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:51.788178 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.788136 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:51.788178 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:51.788166 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:02:52.460500 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:52.460456 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerStarted","Data":"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8"} Jul 22 05:02:52.460696 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:52.460513 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerStarted","Data":"b18ccf9c89477034d55af3adc110f007ea58d37ea310dd1a7c564065ec746f65"} Jul 22 05:02:52.481379 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:52.481335 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podStartSLOduration=1.481318419 podStartE2EDuration="1.481318419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:02:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:02:52.479550126 +0000 UTC m=+1787.527818387" watchObservedRunningTime="2026-07-22 05:02:52.481318419 +0000 UTC m=+1787.529586713" Jul 22 05:02:52.645598 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:52.645571 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:52.650222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:52.650200 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:53.465512 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:53.465474 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:53.466442 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:53.466420 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:02:55.865121 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:55.865054 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:56.054077 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054040 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bng7n\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054077 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054083 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054174 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054196 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054212 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054238 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054273 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054304 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054332 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo\") pod \"602994c1-640e-46ba-a9c5-aa49be6cf943\" (UID: \"602994c1-640e-46ba-a9c5-aa49be6cf943\") " Jul 22 05:02:56.054716 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054447 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:56.054716 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054527 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:56.054716 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054544 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:02:56.054716 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054548 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:56.054933 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054727 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.054933 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054749 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/602994c1-640e-46ba-a9c5-aa49be6cf943-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.054933 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054763 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.054933 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.054779 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.055217 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.055194 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data" (OuterVolumeSpecName: "istio-data") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:56.056510 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.056484 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token" (OuterVolumeSpecName: "istio-token") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:02:56.056629 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.056591 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:02:56.056803 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.056778 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:02:56.056899 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.056855 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n" (OuterVolumeSpecName: "kube-api-access-bng7n") pod "602994c1-640e-46ba-a9c5-aa49be6cf943" (UID: "602994c1-640e-46ba-a9c5-aa49be6cf943"). InnerVolumeSpecName "kube-api-access-bng7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:02:56.156185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.156117 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bng7n\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-kube-api-access-bng7n\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.156185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.156142 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.156185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.156153 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.156185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.156161 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.156185 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.156170 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/602994c1-640e-46ba-a9c5-aa49be6cf943-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:02:56.482544 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.482468 2555 generic.go:356] "Generic (PLEG): container finished" podID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerID="4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02" exitCode=0 Jul 22 05:02:56.482544 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.482516 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" Jul 22 05:02:56.482749 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.482555 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerDied","Data":"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02"} Jul 22 05:02:56.482749 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.482616 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk" event={"ID":"602994c1-640e-46ba-a9c5-aa49be6cf943","Type":"ContainerDied","Data":"63818b00e8176b65678aa9bb0dffc553abca13578fe72b807e3503e2e401fd5f"} Jul 22 05:02:56.482749 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.482651 2555 scope.go:117] "RemoveContainer" containerID="4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02" Jul 22 05:02:56.492753 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.492735 2555 scope.go:117] "RemoveContainer" containerID="e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4" Jul 22 05:02:56.500995 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.500978 2555 scope.go:117] "RemoveContainer" containerID="4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02" Jul 22 05:02:56.501252 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:02:56.501233 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02\": container with ID starting with 4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02 not found: ID does not exist" containerID="4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02" Jul 22 05:02:56.501342 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.501263 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02"} err="failed to get container status \"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02\": rpc error: code = NotFound desc = could not find container \"4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02\": container with ID starting with 4922ad39e89a2dce42a452dd45215ddb246c1b4288e94404dd4a46cd8e105f02 not found: ID does not exist" Jul 22 05:02:56.501342 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.501286 2555 scope.go:117] "RemoveContainer" containerID="e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4" Jul 22 05:02:56.501534 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:02:56.501518 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4\": container with ID starting with e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4 not found: ID does not exist" containerID="e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4" Jul 22 05:02:56.501573 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.501539 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4"} err="failed to get container status \"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4\": rpc error: code = NotFound desc = could not find container \"e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4\": container with ID starting with e4d918d151e5f5f2449bbb3fc49041098a20325c9cef6f11f95915ad7c3ba4e4 not found: ID does not exist" Jul 22 05:02:56.507271 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.507242 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:56.510012 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:56.509993 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-303fe40d-openshift-default-7fcc85b6c7-m5lsk"] Jul 22 05:02:57.431163 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:02:57.431088 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" path="/var/lib/kubelet/pods/602994c1-640e-46ba-a9c5-aa49be6cf943/volumes" Jul 22 05:03:05.482233 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:05.482206 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:03:05.487824 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:05.487802 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:03:08.466937 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.466905 2555 patch_prober.go:28] interesting pod/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 05:03:08.467313 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.466994 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 05:03:08.555919 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.555865 2555 generic.go:356] "Generic (PLEG): container finished" podID="d1f43116-5068-4d26-af12-042f0f726ae3" containerID="a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8" exitCode=0 Jul 22 05:03:08.556049 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.555942 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerDied","Data":"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8"} Jul 22 05:03:08.556363 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.556347 2555 scope.go:117] "RemoveContainer" containerID="a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8" Jul 22 05:03:08.557369 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.557346 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:03:08.557424 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.557407 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:03:08.557460 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:08.557442 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:03:09.564385 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:09.564350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerStarted","Data":"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd"} Jul 22 05:03:09.644979 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:09.644938 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:09.645186 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:09.645162 2555 patch_prober.go:28] interesting pod/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 05:03:09.645236 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:09.645215 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 05:03:10.645427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:10.645389 2555 patch_prober.go:28] interesting pod/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 05:03:10.645888 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:10.645448 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 05:03:11.645925 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:11.645893 2555 patch_prober.go:28] interesting pod/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 05:03:11.646326 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:11.645942 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 05:03:12.645767 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:12.645734 2555 patch_prober.go:28] interesting pod/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 22 05:03:12.645914 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:12.645784 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 22 05:03:13.648833 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:13.648758 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:13.649181 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:13.649035 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:13.649708 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:13.649690 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:33.914312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:33.914276 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:03:33.914902 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:33.914537 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" containerID="cri-o://50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd" gracePeriod=30 Jul 22 05:03:39.169675 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.169649 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:39.257692 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257666 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.257811 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257768 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.257871 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257827 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhdtb\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.257917 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257869 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.257973 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257941 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.257973 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257964 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.258106 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.257988 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.258106 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258014 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.258258 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258233 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:03:39.258325 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258290 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:03:39.258382 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258315 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:03:39.258465 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258441 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.258586 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258467 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.258586 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258483 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.259047 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.258998 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data" (OuterVolumeSpecName: "istio-data") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:03:39.260171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.260102 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb" (OuterVolumeSpecName: "kube-api-access-qhdtb") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "kube-api-access-qhdtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:03:39.260291 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.260255 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:03:39.260443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.260423 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token" (OuterVolumeSpecName: "istio-token") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:03:39.260520 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.260447 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:03:39.358840 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.358819 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert\") pod \"d1f43116-5068-4d26-af12-042f0f726ae3\" (UID: \"d1f43116-5068-4d26-af12-042f0f726ae3\") " Jul 22 05:03:39.359103 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359085 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhdtb\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-kube-api-access-qhdtb\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.359162 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359113 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.359162 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359140 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1f43116-5068-4d26-af12-042f0f726ae3-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.359162 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359155 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1f43116-5068-4d26-af12-042f0f726ae3-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.359263 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359168 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1f43116-5068-4d26-af12-042f0f726ae3-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.359263 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.359209 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d1f43116-5068-4d26-af12-042f0f726ae3" (UID: "d1f43116-5068-4d26-af12-042f0f726ae3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:03:39.463388 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.460752 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1f43116-5068-4d26-af12-042f0f726ae3-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:03:39.715507 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.715426 2555 generic.go:356] "Generic (PLEG): container finished" podID="d1f43116-5068-4d26-af12-042f0f726ae3" containerID="50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd" exitCode=0 Jul 22 05:03:39.715507 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.715487 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" Jul 22 05:03:39.715507 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.715501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerDied","Data":"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd"} Jul 22 05:03:39.715764 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.715573 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj" event={"ID":"d1f43116-5068-4d26-af12-042f0f726ae3","Type":"ContainerDied","Data":"b18ccf9c89477034d55af3adc110f007ea58d37ea310dd1a7c564065ec746f65"} Jul 22 05:03:39.715764 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.715595 2555 scope.go:117] "RemoveContainer" containerID="50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd" Jul 22 05:03:39.724008 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.723990 2555 scope.go:117] "RemoveContainer" containerID="a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8" Jul 22 05:03:39.731478 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.731459 2555 scope.go:117] "RemoveContainer" containerID="50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd" Jul 22 05:03:39.731720 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:03:39.731703 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd\": container with ID starting with 50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd not found: ID does not exist" containerID="50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd" Jul 22 05:03:39.731777 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.731729 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd"} err="failed to get container status \"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd\": rpc error: code = NotFound desc = could not find container \"50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd\": container with ID starting with 50ab463446bc69bdf60d5e79cc01b27cd291736373de3c520a35dde4c998a1fd not found: ID does not exist" Jul 22 05:03:39.731777 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.731745 2555 scope.go:117] "RemoveContainer" containerID="a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8" Jul 22 05:03:39.731984 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:03:39.731966 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8\": container with ID starting with a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8 not found: ID does not exist" containerID="a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8" Jul 22 05:03:39.732051 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.731990 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8"} err="failed to get container status \"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8\": rpc error: code = NotFound desc = could not find container \"a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8\": container with ID starting with a03f03989cf537aa36ddb03bc60660fdbd88fb97f8f2fab25dde7a144edd46e8 not found: ID does not exist" Jul 22 05:03:39.733361 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.733341 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:03:39.736686 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:39.736665 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9e622b09-openshift-default-5c44b95955-w7hvj"] Jul 22 05:03:41.432113 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:03:41.432081 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" path="/var/lib/kubelet/pods/d1f43116-5068-4d26-af12-042f0f726ae3/volumes" Jul 22 05:05:09.775905 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.775866 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776318 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776332 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776345 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776350 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776365 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776370 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776389 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776396 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:05:09.776404 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776408 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:05:09.776728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776416 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:05:09.776728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776487 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:05:09.776728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776495 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:05:09.776728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.776502 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfebfa47-32a4-42ab-a94b-2aa202202456" containerName="istio-proxy" Jul 22 05:05:09.779948 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.779924 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.782505 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.782483 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-31b554fe-openshift-default-dockercfg-jzdth\"" Jul 22 05:05:09.806503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.806477 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:09.927632 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.927632 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.927893 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs7fm\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.927893 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927754 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.927893 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927789 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.927893 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927876 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.928152 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.928152 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927927 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:09.928152 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:09.927981 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029132 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029200 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029237 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029301 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029343 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029331 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029660 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029370 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs7fm\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029660 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029498 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029770 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029732 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029827 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.029890 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029859 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.030013 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.029990 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.031542 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.031481 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.032079 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.032053 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.036700 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.036676 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.037122 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.037012 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs7fm\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm\") pod \"e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.090431 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.090402 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:10.436635 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.436610 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:10.438727 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:05:10.438698 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633b0dc0_400b_403e_9cf8_3cb5ff5bfd76.slice/crio-60062be26198d2c74e37ab698c8cb0039d6cd3dcc546e1fb9fa76344636a10a4 WatchSource:0}: Error finding container 60062be26198d2c74e37ab698c8cb0039d6cd3dcc546e1fb9fa76344636a10a4: Status 404 returned error can't find the container with id 60062be26198d2c74e37ab698c8cb0039d6cd3dcc546e1fb9fa76344636a10a4 Jul 22 05:05:10.440666 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.440633 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:10.440751 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.440704 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:10.440751 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:10.440732 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:11.179419 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:11.179381 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerStarted","Data":"5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727"} Jul 22 05:05:11.179780 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:11.179437 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerStarted","Data":"60062be26198d2c74e37ab698c8cb0039d6cd3dcc546e1fb9fa76344636a10a4"} Jul 22 05:05:11.198080 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:11.198010 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" podStartSLOduration=2.197991782 podStartE2EDuration="2.197991782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:05:11.196714788 +0000 UTC m=+1926.244983046" watchObservedRunningTime="2026-07-22 05:05:11.197991782 +0000 UTC m=+1926.246260040" Jul 22 05:05:12.090829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:12.090792 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:12.095702 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:12.095664 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:12.186734 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:12.186694 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:12.187739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:12.187713 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:24.264456 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.264423 2555 generic.go:356] "Generic (PLEG): container finished" podID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerID="5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727" exitCode=0 Jul 22 05:05:24.264876 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.264490 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerDied","Data":"5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727"} Jul 22 05:05:24.265015 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.264997 2555 scope.go:117] "RemoveContainer" containerID="5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727" Jul 22 05:05:24.266087 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.266061 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:24.266218 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.266127 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:24.266218 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:24.266168 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:05:25.271748 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:25.271718 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerStarted","Data":"d33aa1a8d049b1ea7c1f5db0178a6bcd31f91360aad533abf85dea027e868b57"} Jul 22 05:05:26.090603 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:26.090563 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:26.092039 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:26.091960 2555 patch_prober.go:28] interesting pod/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 05:05:26.092206 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:26.092069 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 05:05:27.091259 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:27.091217 2555 patch_prober.go:28] interesting pod/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 05:05:27.091730 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:27.091296 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 05:05:27.187266 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:27.187221 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:28.090904 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:28.090870 2555 patch_prober.go:28] interesting pod/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 22 05:05:28.091112 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:28.090934 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 22 05:05:29.095145 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:29.095115 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:29.096103 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:29.096080 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:46.160802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:46.160771 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:46.161331 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:46.160978 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" containerID="cri-o://d33aa1a8d049b1ea7c1f5db0178a6bcd31f91360aad533abf85dea027e868b57" gracePeriod=30 Jul 22 05:05:51.406387 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.406359 2555 generic.go:356] "Generic (PLEG): container finished" podID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerID="d33aa1a8d049b1ea7c1f5db0178a6bcd31f91360aad533abf85dea027e868b57" exitCode=0 Jul 22 05:05:51.406789 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.406451 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerDied","Data":"d33aa1a8d049b1ea7c1f5db0178a6bcd31f91360aad533abf85dea027e868b57"} Jul 22 05:05:51.406789 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.406511 2555 scope.go:117] "RemoveContainer" containerID="5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727" Jul 22 05:05:51.420832 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:05:51.420793 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727\": container with ID starting with 5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727 not found: ID does not exist" containerID="5bc4c6765f135b860fa65ea5190c27aabc44ff84f11aad0d694c7ee6b635a727" Jul 22 05:05:51.424284 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.424262 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:51.524563 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524534 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524712 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524600 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524712 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524630 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524712 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524667 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524842 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524724 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xs7fm\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524842 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524761 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524842 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524784 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.524842 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524826 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.525093 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.524875 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket\") pod \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\" (UID: \"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76\") " Jul 22 05:05:51.525093 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525009 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:05:51.525093 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525050 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:05:51.525276 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525243 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:05:51.525388 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525370 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.525455 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525418 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.525455 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525435 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.525564 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525459 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:05:51.525624 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.525608 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data" (OuterVolumeSpecName: "istio-data") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:05:51.526839 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.526810 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token" (OuterVolumeSpecName: "istio-token") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:05:51.526993 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.526971 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm" (OuterVolumeSpecName: "kube-api-access-xs7fm") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "kube-api-access-xs7fm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:05:51.527093 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.527075 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:05:51.527223 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.527199 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" (UID: "633b0dc0-400b-403e-9cf8-3cb5ff5bfd76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:05:51.626846 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626815 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xs7fm\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-kube-api-access-xs7fm\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.626846 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626840 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.626846 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626850 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.627090 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626858 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.627090 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626866 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:51.627090 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:51.626873 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:05:52.412997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:52.412958 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" event={"ID":"633b0dc0-400b-403e-9cf8-3cb5ff5bfd76","Type":"ContainerDied","Data":"60062be26198d2c74e37ab698c8cb0039d6cd3dcc546e1fb9fa76344636a10a4"} Jul 22 05:05:52.413410 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:52.413043 2555 scope.go:117] "RemoveContainer" containerID="d33aa1a8d049b1ea7c1f5db0178a6bcd31f91360aad533abf85dea027e868b57" Jul 22 05:05:52.413410 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:52.412971 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd" Jul 22 05:05:52.434605 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:52.434581 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:52.438849 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:52.438827 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-31b554fe-openshift-default-c4c4678f9-4bpsd"] Jul 22 05:05:53.430610 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:05:53.430574 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" path="/var/lib/kubelet/pods/633b0dc0-400b-403e-9cf8-3cb5ff5bfd76/volumes" Jul 22 05:06:05.260802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.259662 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:06:05.260802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.260772 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:06:05.260802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.260795 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:06:05.261425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.260824 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:06:05.261425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.260835 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:06:05.261425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.261063 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:06:05.261425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.261083 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="602994c1-640e-46ba-a9c5-aa49be6cf943" containerName="istio-proxy" Jul 22 05:06:05.261425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.261103 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1f43116-5068-4d26-af12-042f0f726ae3" containerName="istio-proxy" Jul 22 05:06:05.267071 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.267046 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.269549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.269520 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-06f9a996-openshift-default-dockercfg-ffbpw\"" Jul 22 05:06:05.271830 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.271800 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:06:05.349066 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349013 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349075 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349122 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349140 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349179 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349222 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349202 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349437 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349254 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbg4z\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349437 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349274 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.349437 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.349292 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450727 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450679 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450794 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450824 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450858 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.450928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450892 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451225 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450946 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbg4z\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451225 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.450977 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451225 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451006 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451225 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451197 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451390 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451255 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451390 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451280 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451462 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451425 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.451495 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.451466 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.453308 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.453289 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.453377 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.453307 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.458457 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.458427 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.458617 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.458547 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbg4z\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z\") pod \"e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.582573 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.582482 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-06f9a996-openshift-default-dockercfg-ffbpw\"" Jul 22 05:06:05.591015 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.590987 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:05.735792 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.735767 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:06:05.737802 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:06:05.737774 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6179d8_4e43_433e_a506_40a7084c3b8e.slice/crio-64407151973c04a722bd8ecdc399495308f8f946274d8e15b79431eeffe0ab5f WatchSource:0}: Error finding container 64407151973c04a722bd8ecdc399495308f8f946274d8e15b79431eeffe0ab5f: Status 404 returned error can't find the container with id 64407151973c04a722bd8ecdc399495308f8f946274d8e15b79431eeffe0ab5f Jul 22 05:06:05.740084 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.740045 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:05.740171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.740140 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:05.740218 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:05.740182 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:06.486489 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:06.486440 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerStarted","Data":"22f6d6a899784cbca9c3c1c936c82c309df45f8c58896ebe667a0e38ff7143d1"} Jul 22 05:06:06.486918 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:06.486503 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerStarted","Data":"64407151973c04a722bd8ecdc399495308f8f946274d8e15b79431eeffe0ab5f"} Jul 22 05:06:06.506148 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:06.506086 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podStartSLOduration=1.506070218 podStartE2EDuration="1.506070218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:06:06.504096259 +0000 UTC m=+1981.552364517" watchObservedRunningTime="2026-07-22 05:06:06.506070218 +0000 UTC m=+1981.554338477" Jul 22 05:06:06.592129 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:06.592098 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:06.597141 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:06.597117 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:07.491568 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:07.491537 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:07.492447 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:07.492428 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:22.831670 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:22.831531 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:38822->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 22 05:06:22.831670 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:22.831621 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:38822->10.132.0.60:15021: read: connection reset by peer" Jul 22 05:06:23.577475 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.577439 2555 generic.go:356] "Generic (PLEG): container finished" podID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerID="22f6d6a899784cbca9c3c1c936c82c309df45f8c58896ebe667a0e38ff7143d1" exitCode=0 Jul 22 05:06:23.577668 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.577482 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerDied","Data":"22f6d6a899784cbca9c3c1c936c82c309df45f8c58896ebe667a0e38ff7143d1"} Jul 22 05:06:23.577900 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.577869 2555 scope.go:117] "RemoveContainer" containerID="22f6d6a899784cbca9c3c1c936c82c309df45f8c58896ebe667a0e38ff7143d1" Jul 22 05:06:23.578885 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.578855 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:23.579004 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.578936 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:23.579004 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:23.578986 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:06:24.584142 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:24.584103 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerStarted","Data":"29e9cd2a9e56ad195c0f57ca06eb5f00ebdc74b8f77344afe3a748cdc465a02e"} Jul 22 05:06:24.591844 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:24.591818 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:24.591994 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:24.591974 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 05:06:24.592079 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:24.592061 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 05:06:25.592279 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:25.592243 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 05:06:25.592656 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:25.592296 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 05:06:26.592353 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:26.592316 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 05:06:26.592721 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:26.592380 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 05:06:27.591710 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:27.591674 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 22 05:06:27.591892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:27.591723 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 22 05:06:28.595282 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:28.595246 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:28.595758 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:28.595651 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:06:28.596386 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:06:28.596364 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:07:01.695807 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.695769 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw"] Jul 22 05:07:01.696536 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.696515 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="633b0dc0-400b-403e-9cf8-3cb5ff5bfd76" containerName="istio-proxy" Jul 22 05:07:01.700248 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.700227 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:01.702971 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.702951 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 05:07:01.703067 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.703011 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4n2k7\"" Jul 22 05:07:01.707453 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.707397 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw"] Jul 22 05:07:01.733627 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.733597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw\" (UID: \"4cfbeb77-662c-48f7-85fb-8185e3cd6a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:01.834309 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.834282 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw\" (UID: \"4cfbeb77-662c-48f7-85fb-8185e3cd6a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:01.834888 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:01.834864 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw\" (UID: \"4cfbeb77-662c-48f7-85fb-8185e3cd6a97\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:02.016144 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:02.016119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:02.354989 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:02.354966 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw"] Jul 22 05:07:02.359986 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:07:02.357333 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfbeb77_662c_48f7_85fb_8185e3cd6a97.slice/crio-e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5 WatchSource:0}: Error finding container e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5: Status 404 returned error can't find the container with id e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5 Jul 22 05:07:02.361569 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:02.361551 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 05:07:02.774978 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:02.774941 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" event={"ID":"4cfbeb77-662c-48f7-85fb-8185e3cd6a97","Type":"ContainerStarted","Data":"e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5"} Jul 22 05:07:04.786957 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:04.786926 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" event={"ID":"4cfbeb77-662c-48f7-85fb-8185e3cd6a97","Type":"ContainerStarted","Data":"a82f4e35bc419933a600fd0ea49fe4b66211ed2b4ca1ae2128c332aa2d03dcd0"} Jul 22 05:07:04.803785 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:04.803741 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" podStartSLOduration=1.865837608 podStartE2EDuration="3.803724511s" totalImagesPullingTime="1.937886903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:07:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:07:04.801784354 +0000 UTC m=+2039.850052613" watchObservedRunningTime="2026-07-22 05:07:04.803724511 +0000 UTC m=+2039.851992814" Jul 22 05:07:05.792865 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:05.792833 2555 generic.go:356] "Generic (PLEG): container finished" podID="4cfbeb77-662c-48f7-85fb-8185e3cd6a97" containerID="a82f4e35bc419933a600fd0ea49fe4b66211ed2b4ca1ae2128c332aa2d03dcd0" exitCode=0 Jul 22 05:07:05.793249 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:05.792895 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" event={"ID":"4cfbeb77-662c-48f7-85fb-8185e3cd6a97","Type":"ContainerDied","Data":"a82f4e35bc419933a600fd0ea49fe4b66211ed2b4ca1ae2128c332aa2d03dcd0"} Jul 22 05:07:06.928695 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:06.928666 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:06.983066 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:06.983011 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca\") pod \"4cfbeb77-662c-48f7-85fb-8185e3cd6a97\" (UID: \"4cfbeb77-662c-48f7-85fb-8185e3cd6a97\") " Jul 22 05:07:06.983311 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:06.983288 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cfbeb77-662c-48f7-85fb-8185e3cd6a97" (UID: "4cfbeb77-662c-48f7-85fb-8185e3cd6a97"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:07:07.083827 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:07.083771 2555 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cfbeb77-662c-48f7-85fb-8185e3cd6a97-maas-api-service-ca\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:07:07.804809 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:07.804776 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" Jul 22 05:07:07.804994 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:07.804798 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw" event={"ID":"4cfbeb77-662c-48f7-85fb-8185e3cd6a97","Type":"ContainerDied","Data":"e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5"} Jul 22 05:07:07.804994 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:07.804898 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e2f704e67540b99a1884b36190d2451569a17a191f640f90bdd75f2e627e2fa5" Jul 22 05:07:21.780399 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:21.780363 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw"] Jul 22 05:07:21.783674 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:21.783652 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-06f9a996-dqhjw"] Jul 22 05:07:23.431085 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:07:23.431048 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cfbeb77-662c-48f7-85fb-8185e3cd6a97" path="/var/lib/kubelet/pods/4cfbeb77-662c-48f7-85fb-8185e3cd6a97/volumes" Jul 22 05:08:05.513705 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:05.513680 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:08:05.526061 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:05.526039 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:08:12.020004 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:12.019956 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:08:12.020695 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:12.020251 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" containerID="cri-o://29e9cd2a9e56ad195c0f57ca06eb5f00ebdc74b8f77344afe3a748cdc465a02e" gracePeriod=30 Jul 22 05:08:13.598417 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:13.598373 2555 patch_prober.go:28] interesting pod/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 05:08:13.598776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:13.598439 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 05:08:17.173107 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.173068 2555 generic.go:356] "Generic (PLEG): container finished" podID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerID="29e9cd2a9e56ad195c0f57ca06eb5f00ebdc74b8f77344afe3a748cdc465a02e" exitCode=0 Jul 22 05:08:17.173495 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.173142 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerDied","Data":"29e9cd2a9e56ad195c0f57ca06eb5f00ebdc74b8f77344afe3a748cdc465a02e"} Jul 22 05:08:17.173495 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.173195 2555 scope.go:117] "RemoveContainer" containerID="22f6d6a899784cbca9c3c1c936c82c309df45f8c58896ebe667a0e38ff7143d1" Jul 22 05:08:17.290756 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.290694 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:08:17.359043 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.358804 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359043 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.358867 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbg4z\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359278 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359246 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359350 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359311 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359350 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359343 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359458 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359377 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359458 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359417 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359481 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.359566 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359523 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs\") pod \"ce6179d8-4e43-433e-a506-40a7084c3b8e\" (UID: \"ce6179d8-4e43-433e-a506-40a7084c3b8e\") " Jul 22 05:08:17.360592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.359967 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:08:17.360592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.360072 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:08:17.360592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.360280 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:08:17.360592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.360321 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:08:17.364064 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.364039 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:08:17.365242 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.364410 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:08:17.365242 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.364486 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:08:17.365242 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.365200 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:08:17.365570 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.365539 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z" (OuterVolumeSpecName: "kube-api-access-rbg4z") pod "ce6179d8-4e43-433e-a506-40a7084c3b8e" (UID: "ce6179d8-4e43-433e-a506-40a7084c3b8e"). InnerVolumeSpecName "kube-api-access-rbg4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:08:17.460504 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460475 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460504 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460502 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460511 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460521 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbg4z\" (UniqueName: \"kubernetes.io/projected/ce6179d8-4e43-433e-a506-40a7084c3b8e-kube-api-access-rbg4z\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460531 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6179d8-4e43-433e-a506-40a7084c3b8e-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460538 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460546 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460553 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:17.460719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:17.460561 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6179d8-4e43-433e-a506-40a7084c3b8e-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:08:18.179157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:18.179113 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" Jul 22 05:08:18.179157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:18.179145 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts" event={"ID":"ce6179d8-4e43-433e-a506-40a7084c3b8e","Type":"ContainerDied","Data":"64407151973c04a722bd8ecdc399495308f8f946274d8e15b79431eeffe0ab5f"} Jul 22 05:08:18.179663 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:18.179207 2555 scope.go:117] "RemoveContainer" containerID="29e9cd2a9e56ad195c0f57ca06eb5f00ebdc74b8f77344afe3a748cdc465a02e" Jul 22 05:08:18.197969 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:18.197943 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:08:18.203537 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:18.203507 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-06f9a996-openshift-default-c7bb8cd7f-tzlts"] Jul 22 05:08:19.430735 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:08:19.430704 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" path="/var/lib/kubelet/pods/ce6179d8-4e43-433e-a506-40a7084c3b8e/volumes" Jul 22 05:09:06.956992 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.956952 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:09:06.957620 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957598 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:06.957710 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957622 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:06.957710 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957651 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cfbeb77-662c-48f7-85fb-8185e3cd6a97" containerName="revoke-keys" Jul 22 05:09:06.957710 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957660 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cfbeb77-662c-48f7-85fb-8185e3cd6a97" containerName="revoke-keys" Jul 22 05:09:06.957866 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957768 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:06.957866 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.957781 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cfbeb77-662c-48f7-85fb-8185e3cd6a97" containerName="revoke-keys" Jul 22 05:09:06.962067 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.962047 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:06.964601 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.964578 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-fce892-openshift-default-dockercfg-hjgqq\"" Jul 22 05:09:06.972668 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:06.972602 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:09:07.089468 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089434 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089491 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089549 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089593 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089612 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089634 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089833 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089739 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089833 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089787 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q59rs\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.089899 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.089837 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190305 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190275 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190312 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190349 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190365 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190461 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190430 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190514 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190572 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q59rs\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190636 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190903 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190757 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190903 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190848 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.190903 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190880 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.191092 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.190929 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.191221 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.191198 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.192956 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.192933 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.193197 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.193177 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.197823 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.197797 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.197914 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.197874 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q59rs\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs\") pod \"e2e-inf-a-fce892-openshift-default-559f57c999-768cs\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.273383 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.273360 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:07.411224 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.411188 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:09:07.412264 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:09:07.412235 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9d0f99_07bb_447f_83b4_ca32fe1454ca.slice/crio-b1f5e18c638031e2b0e42c840d2c40cce52492170dbb22a0769960d7cd232da6 WatchSource:0}: Error finding container b1f5e18c638031e2b0e42c840d2c40cce52492170dbb22a0769960d7cd232da6: Status 404 returned error can't find the container with id b1f5e18c638031e2b0e42c840d2c40cce52492170dbb22a0769960d7cd232da6 Jul 22 05:09:07.414595 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.414561 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:07.414681 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.414646 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:07.414720 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.414687 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:07.444550 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:07.444526 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerStarted","Data":"b1f5e18c638031e2b0e42c840d2c40cce52492170dbb22a0769960d7cd232da6"} Jul 22 05:09:08.451980 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:08.451944 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerStarted","Data":"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b"} Jul 22 05:09:08.473491 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:08.473446 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podStartSLOduration=2.473432242 podStartE2EDuration="2.473432242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:09:08.470545328 +0000 UTC m=+2163.518813587" watchObservedRunningTime="2026-07-22 05:09:08.473432242 +0000 UTC m=+2163.521700500" Jul 22 05:09:09.274170 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:09.274131 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:09.279083 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:09.279058 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:09.456672 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:09.456645 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:09.457402 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:09.457382 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:15.535196 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.535162 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:09:15.535603 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.535590 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:15.535647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.535605 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:15.535689 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.535680 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce6179d8-4e43-433e-a506-40a7084c3b8e" containerName="istio-proxy" Jul 22 05:09:15.538960 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.538940 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.541201 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.541175 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8e4aa9-openshift-default-dockercfg-lq7rn\"" Jul 22 05:09:15.548650 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.548631 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:09:15.664651 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664624 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664785 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664659 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcbqg\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664785 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664678 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664785 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664772 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664894 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664894 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664959 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664896 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664959 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.664959 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.664943 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.765898 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.765867 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbqg\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.765898 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.765898 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.765954 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.765977 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766005 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766103 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766131 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766453 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766216 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766537 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766510 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766616 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766596 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766677 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766650 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766728 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766670 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.766802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.766780 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.768520 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.768503 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.768666 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.768647 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.773191 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.773161 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbqg\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.773191 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.773163 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token\") pod \"e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:15.851447 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:15.851379 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:16.007892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.007857 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:09:16.010509 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:09:16.010480 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d64b73_4be2_468a_9b81_474a90f287de.slice/crio-cdab558de5b737e8f569bab0e18f8cc5f123e4e5e48f19caabba7c69b18901bf WatchSource:0}: Error finding container cdab558de5b737e8f569bab0e18f8cc5f123e4e5e48f19caabba7c69b18901bf: Status 404 returned error can't find the container with id cdab558de5b737e8f569bab0e18f8cc5f123e4e5e48f19caabba7c69b18901bf Jul 22 05:09:16.012785 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.012743 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:16.012906 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.012833 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:16.012906 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.012882 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:16.494338 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.494306 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerStarted","Data":"fc99e23682e8c573082cf81a08e1a17c686bbce3a957210b5a9d1e034383820e"} Jul 22 05:09:16.494495 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.494360 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerStarted","Data":"cdab558de5b737e8f569bab0e18f8cc5f123e4e5e48f19caabba7c69b18901bf"} Jul 22 05:09:16.514911 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.514857 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podStartSLOduration=1.514841459 podStartE2EDuration="1.514841459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:09:16.51296023 +0000 UTC m=+2171.561228491" watchObservedRunningTime="2026-07-22 05:09:16.514841459 +0000 UTC m=+2171.563109717" Jul 22 05:09:16.851918 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:16.851866 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:17.858527 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:17.858496 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:18.511133 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:18.511099 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:18.543776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:18.543351 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:24.458432 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.458313 2555 patch_prober.go:28] interesting pod/e2e-inf-a-fce892-openshift-default-559f57c999-768cs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 05:09:24.458432 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.458394 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 05:09:24.554252 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.554216 2555 generic.go:356] "Generic (PLEG): container finished" podID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerID="99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b" exitCode=0 Jul 22 05:09:24.554443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.554292 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerDied","Data":"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b"} Jul 22 05:09:24.554796 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.554777 2555 scope.go:117] "RemoveContainer" containerID="99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b" Jul 22 05:09:24.555768 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.555741 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:24.555821 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.555811 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:24.555868 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:24.555847 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:25.567429 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:25.567391 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerStarted","Data":"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54"} Jul 22 05:09:26.274414 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:26.274381 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:26.274597 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:26.274482 2555 patch_prober.go:28] interesting pod/e2e-inf-a-fce892-openshift-default-559f57c999-768cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 05:09:26.274597 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:26.274530 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 05:09:27.274083 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:27.274006 2555 patch_prober.go:28] interesting pod/e2e-inf-a-fce892-openshift-default-559f57c999-768cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 05:09:27.274502 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:27.274100 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 05:09:28.274938 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:28.274836 2555 patch_prober.go:28] interesting pod/e2e-inf-a-fce892-openshift-default-559f57c999-768cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 05:09:28.274938 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:28.274909 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 05:09:29.273722 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:29.273684 2555 patch_prober.go:28] interesting pod/e2e-inf-a-fce892-openshift-default-559f57c999-768cs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 22 05:09:29.273942 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:29.273755 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 22 05:09:30.278466 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:30.278429 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:30.278954 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:30.278840 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:30.279773 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:30.279755 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:09:31.602277 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.602183 2555 generic.go:356] "Generic (PLEG): container finished" podID="a5d64b73-4be2-468a-9b81-474a90f287de" containerID="fc99e23682e8c573082cf81a08e1a17c686bbce3a957210b5a9d1e034383820e" exitCode=0 Jul 22 05:09:31.602277 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.602261 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerDied","Data":"fc99e23682e8c573082cf81a08e1a17c686bbce3a957210b5a9d1e034383820e"} Jul 22 05:09:31.602803 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.602790 2555 scope.go:117] "RemoveContainer" containerID="fc99e23682e8c573082cf81a08e1a17c686bbce3a957210b5a9d1e034383820e" Jul 22 05:09:31.604063 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.603997 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:31.604164 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.604125 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:31.604234 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:31.604174 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:09:32.613392 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:32.613353 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerStarted","Data":"27b7798b35513beb1417b39d84bb84f0e96ef4f8327428fda7a43f6478572f0e"} Jul 22 05:09:32.852519 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:32.852478 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:32.853767 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:32.853740 2555 patch_prober.go:28] interesting pod/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 05:09:32.853904 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:32.853792 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 05:09:33.511397 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:33.511335 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:33.852163 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:33.852071 2555 patch_prober.go:28] interesting pod/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 05:09:33.852163 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:33.852131 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 05:09:34.852303 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:34.852264 2555 patch_prober.go:28] interesting pod/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 05:09:34.852696 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:34.852338 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 05:09:35.851915 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:35.851875 2555 patch_prober.go:28] interesting pod/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 22 05:09:35.852119 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:35.851944 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 22 05:09:36.855638 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:36.855607 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:09:36.856740 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:09:36.856717 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:11:05.096412 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:05.096377 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:11:05.097000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:05.096626 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" containerID="cri-o://fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54" gracePeriod=30 Jul 22 05:11:10.363565 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.363496 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:11:10.410262 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410226 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410300 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410332 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410384 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410409 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q59rs\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410443 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410436 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410465 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410539 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410560 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:10.410709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410566 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token\") pod \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\" (UID: \"4b9d0f99-07bb-447f-83b4-ca32fe1454ca\") " Jul 22 05:11:10.410709 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410662 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:10.410984 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.410795 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:11:10.411116 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411078 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:10.411275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411201 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.411275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411221 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.411275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411235 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.411275 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411249 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.411582 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.411555 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:10.413265 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.413227 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:11:10.413265 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.413242 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:11:10.413394 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.413282 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs" (OuterVolumeSpecName: "kube-api-access-q59rs") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "kube-api-access-q59rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:11:10.413394 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.413313 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b9d0f99-07bb-447f-83b4-ca32fe1454ca" (UID: "4b9d0f99-07bb-447f-83b4-ca32fe1454ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:10.512131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.512094 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q59rs\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-kube-api-access-q59rs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.512131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.512124 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.512131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.512133 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.512131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.512141 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:10.512402 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:10.512149 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b9d0f99-07bb-447f-83b4-ca32fe1454ca-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:11.155567 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.155532 2555 generic.go:356] "Generic (PLEG): container finished" podID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerID="fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54" exitCode=0 Jul 22 05:11:11.155965 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.155584 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" Jul 22 05:11:11.155965 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.155623 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerDied","Data":"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54"} Jul 22 05:11:11.155965 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.155677 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs" event={"ID":"4b9d0f99-07bb-447f-83b4-ca32fe1454ca","Type":"ContainerDied","Data":"b1f5e18c638031e2b0e42c840d2c40cce52492170dbb22a0769960d7cd232da6"} Jul 22 05:11:11.155965 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.155700 2555 scope.go:117] "RemoveContainer" containerID="fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54" Jul 22 05:11:11.165658 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.165641 2555 scope.go:117] "RemoveContainer" containerID="99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b" Jul 22 05:11:11.174249 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.174228 2555 scope.go:117] "RemoveContainer" containerID="fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54" Jul 22 05:11:11.174516 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:11:11.174492 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54\": container with ID starting with fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54 not found: ID does not exist" containerID="fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54" Jul 22 05:11:11.174629 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.174527 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54"} err="failed to get container status \"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54\": rpc error: code = NotFound desc = could not find container \"fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54\": container with ID starting with fdf267b8d98548ef1d43d21aa7725bb0f460544eb2cd6fa1aca6cd48a6d66e54 not found: ID does not exist" Jul 22 05:11:11.174629 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.174546 2555 scope.go:117] "RemoveContainer" containerID="99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b" Jul 22 05:11:11.175080 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:11:11.174955 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b\": container with ID starting with 99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b not found: ID does not exist" containerID="99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b" Jul 22 05:11:11.175080 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.174990 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b"} err="failed to get container status \"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b\": rpc error: code = NotFound desc = could not find container \"99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b\": container with ID starting with 99aa91f470afd011c8b86c32ad99b57969e909d631d051c27c04647a441f4b6b not found: ID does not exist" Jul 22 05:11:11.177699 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.177678 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:11:11.180697 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.180673 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-fce892-openshift-default-559f57c999-768cs"] Jul 22 05:11:11.431875 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:11.431781 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" path="/var/lib/kubelet/pods/4b9d0f99-07bb-447f-83b4-ca32fe1454ca/volumes" Jul 22 05:11:27.046087 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.046043 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:11:27.046542 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.046357 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" containerID="cri-o://27b7798b35513beb1417b39d84bb84f0e96ef4f8327428fda7a43f6478572f0e" gracePeriod=30 Jul 22 05:11:27.990478 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.990437 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:11:27.990884 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.990869 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:27.990884 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.990883 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:27.991064 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.990963 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:27.995669 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.995645 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:27.997949 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:27.997925 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-c3de18-openshift-default-dockercfg-h6kmn\"" Jul 22 05:11:28.003751 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.003535 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:11:28.085083 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085095 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085136 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085154 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085279 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085333 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085432 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdc6\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.085596 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.085477 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186300 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186255 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186327 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186354 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186447 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186503 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186495 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186723 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186549 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186723 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186578 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdc6\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186723 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186638 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186723 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186689 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186924 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186790 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186924 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186796 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.186924 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.186831 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.187186 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.187159 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.187300 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.187221 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.189037 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.189003 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.189153 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.189132 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.196293 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.196261 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.196401 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.196364 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdc6\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6\") pod \"e2e-api-a-c3de18-openshift-default-55bf697595-2758h\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.310009 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.309963 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:28.463135 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.463052 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:11:28.465526 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:11:28.465491 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec283059_a0cf_428f_aa7c_56223b581948.slice/crio-23325fccc0311626e60dcce4cc02eff976321b982cfc36a9d60d0cac9c924475 WatchSource:0}: Error finding container 23325fccc0311626e60dcce4cc02eff976321b982cfc36a9d60d0cac9c924475: Status 404 returned error can't find the container with id 23325fccc0311626e60dcce4cc02eff976321b982cfc36a9d60d0cac9c924475 Jul 22 05:11:28.467739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.467707 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:28.467853 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.467789 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:28.467853 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:28.467837 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:29.261344 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:29.261305 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerStarted","Data":"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969"} Jul 22 05:11:29.261840 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:29.261365 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerStarted","Data":"23325fccc0311626e60dcce4cc02eff976321b982cfc36a9d60d0cac9c924475"} Jul 22 05:11:29.282672 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:29.282613 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podStartSLOduration=2.282595928 podStartE2EDuration="2.282595928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:11:29.279951597 +0000 UTC m=+2304.328219894" watchObservedRunningTime="2026-07-22 05:11:29.282595928 +0000 UTC m=+2304.330864188" Jul 22 05:11:29.310881 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:29.310825 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:29.316409 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:29.316381 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:30.266549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:30.266513 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:30.267647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:30.267627 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:32.279813 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.279781 2555 generic.go:356] "Generic (PLEG): container finished" podID="a5d64b73-4be2-468a-9b81-474a90f287de" containerID="27b7798b35513beb1417b39d84bb84f0e96ef4f8327428fda7a43f6478572f0e" exitCode=0 Jul 22 05:11:32.280306 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.279851 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerDied","Data":"27b7798b35513beb1417b39d84bb84f0e96ef4f8327428fda7a43f6478572f0e"} Jul 22 05:11:32.280306 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.279952 2555 scope.go:117] "RemoveContainer" containerID="fc99e23682e8c573082cf81a08e1a17c686bbce3a957210b5a9d1e034383820e" Jul 22 05:11:32.328229 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.328196 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:11:32.432315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432278 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432315 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432314 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432362 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432385 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432458 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432498 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432531 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432567 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcbqg\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.432647 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432623 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert\") pod \"a5d64b73-4be2-468a-9b81-474a90f287de\" (UID: \"a5d64b73-4be2-468a-9b81-474a90f287de\") " Jul 22 05:11:32.433005 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432669 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:32.433005 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432695 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:32.433005 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.432805 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:32.433187 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.433038 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:11:32.433187 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.433091 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.433360 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.433340 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.433439 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.433366 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.433439 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.433369 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:32.434934 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.434907 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:11:32.435154 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.435083 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:11:32.435347 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.435325 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:11:32.435427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.435354 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg" (OuterVolumeSpecName: "kube-api-access-dcbqg") pod "a5d64b73-4be2-468a-9b81-474a90f287de" (UID: "a5d64b73-4be2-468a-9b81-474a90f287de"). InnerVolumeSpecName "kube-api-access-dcbqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:11:32.534706 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534672 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcbqg\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-kube-api-access-dcbqg\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.534706 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534701 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5d64b73-4be2-468a-9b81-474a90f287de-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.534706 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534713 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.534999 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534725 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5d64b73-4be2-468a-9b81-474a90f287de-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.534999 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534738 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5d64b73-4be2-468a-9b81-474a90f287de-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:32.534999 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:32.534751 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5d64b73-4be2-468a-9b81-474a90f287de-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:11:33.286800 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.286773 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" Jul 22 05:11:33.287308 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.286772 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr" event={"ID":"a5d64b73-4be2-468a-9b81-474a90f287de","Type":"ContainerDied","Data":"cdab558de5b737e8f569bab0e18f8cc5f123e4e5e48f19caabba7c69b18901bf"} Jul 22 05:11:33.287308 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.286921 2555 scope.go:117] "RemoveContainer" containerID="27b7798b35513beb1417b39d84bb84f0e96ef4f8327428fda7a43f6478572f0e" Jul 22 05:11:33.310408 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.310361 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:11:33.312943 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.312911 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8e4aa9-openshift-default-d4fbc8647-mxhqr"] Jul 22 05:11:33.431924 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:33.431887 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" path="/var/lib/kubelet/pods/a5d64b73-4be2-468a-9b81-474a90f287de/volumes" Jul 22 05:11:36.638319 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638281 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638787 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638802 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638815 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638821 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638828 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.638837 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638833 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.639045 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638902 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b9d0f99-07bb-447f-83b4-ca32fe1454ca" containerName="istio-proxy" Jul 22 05:11:36.639045 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638914 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.639045 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.638921 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5d64b73-4be2-468a-9b81-474a90f287de" containerName="istio-proxy" Jul 22 05:11:36.643339 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.643312 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.645751 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.645724 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-04dfa1-openshift-default-dockercfg-tf4b8\"" Jul 22 05:11:36.653528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.653379 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:11:36.774760 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.774717 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.774760 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.774764 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.774783 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.774900 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.774938 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775229 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.775081 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775229 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.775143 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhmtj\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775229 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.775196 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.775379 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.775232 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876464 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876412 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876479 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhmtj\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876522 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876551 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876639 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.876687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876663 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877018 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876737 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877018 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877018 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876879 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877018 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.876998 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877274 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.877099 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877274 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.877177 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.877386 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.877362 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.879000 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.878982 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.879122 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.879103 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.885210 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.885179 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.885354 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.885294 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhmtj\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj\") pod \"e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:36.958928 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:36.958820 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:37.111769 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.111639 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:11:37.114808 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:11:37.114767 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c9c726_ddb8_4d6e_9ad0_55bd0ed48b53.slice/crio-4ddaae4548a3fca30edc41d8464160141ac9a43e309ec4d87ab96e46f44d0731 WatchSource:0}: Error finding container 4ddaae4548a3fca30edc41d8464160141ac9a43e309ec4d87ab96e46f44d0731: Status 404 returned error can't find the container with id 4ddaae4548a3fca30edc41d8464160141ac9a43e309ec4d87ab96e46f44d0731 Jul 22 05:11:37.117035 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.116982 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:37.117174 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.117068 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:37.117174 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.117100 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:37.315657 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.315605 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerStarted","Data":"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc"} Jul 22 05:11:37.315883 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.315682 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerStarted","Data":"4ddaae4548a3fca30edc41d8464160141ac9a43e309ec4d87ab96e46f44d0731"} Jul 22 05:11:37.333695 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.333628 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podStartSLOduration=1.333606423 podStartE2EDuration="1.333606423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:11:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:11:37.332480149 +0000 UTC m=+2312.380748410" watchObservedRunningTime="2026-07-22 05:11:37.333606423 +0000 UTC m=+2312.381874682" Jul 22 05:11:37.959145 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:37.959105 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:38.964525 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:38.964491 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:39.335232 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:39.335201 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:39.337704 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:39.337681 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:45.268446 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.268403 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 05:11:45.268951 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.268492 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 05:11:45.370492 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.370451 2555 generic.go:356] "Generic (PLEG): container finished" podID="ec283059-a0cf-428f-aa7c-56223b581948" containerID="0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969" exitCode=0 Jul 22 05:11:45.370691 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.370500 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerDied","Data":"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969"} Jul 22 05:11:45.370995 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.370978 2555 scope.go:117] "RemoveContainer" containerID="0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969" Jul 22 05:11:45.372100 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.372074 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:45.372162 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.372138 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:45.372203 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:45.372169 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:46.378378 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:46.378340 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerStarted","Data":"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f"} Jul 22 05:11:47.313530 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:47.313482 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:47.313746 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:47.313632 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 05:11:47.313746 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:47.313677 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 05:11:48.310600 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:48.310554 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 05:11:48.311141 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:48.310626 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 05:11:49.311460 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:49.311406 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 05:11:49.311983 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:49.311481 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 05:11:50.310485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:50.310448 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 22 05:11:50.310776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:50.310749 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 22 05:11:51.315470 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:51.315436 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:51.315948 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:51.315814 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:51.316533 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:51.316517 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:11:53.419200 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.419156 2555 generic.go:356] "Generic (PLEG): container finished" podID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerID="c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc" exitCode=0 Jul 22 05:11:53.419819 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.419231 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerDied","Data":"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc"} Jul 22 05:11:53.419819 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.419720 2555 scope.go:117] "RemoveContainer" containerID="c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc" Jul 22 05:11:53.420983 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.420948 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:53.421101 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.421082 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:53.421154 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:53.421138 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:11:54.335918 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:54.335881 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:54.429716 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:54.429671 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerStarted","Data":"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544"} Jul 22 05:11:54.959983 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:54.959946 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:54.961129 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:54.961099 2555 patch_prober.go:28] interesting pod/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 05:11:54.961255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:54.961153 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 05:11:55.959623 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:55.959579 2555 patch_prober.go:28] interesting pod/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 05:11:55.960068 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:55.959661 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 05:11:56.959528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:56.959486 2555 patch_prober.go:28] interesting pod/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 05:11:56.959739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:56.959553 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 05:11:57.960355 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:57.960260 2555 patch_prober.go:28] interesting pod/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 22 05:11:57.960355 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:57.960327 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 22 05:11:58.963640 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:58.963610 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:58.964109 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:58.964052 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:11:58.964832 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:11:58.964813 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:12:18.557137 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:18.557095 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:12:18.557762 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:18.557347 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" containerID="cri-o://da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f" gracePeriod=30 Jul 22 05:12:21.319127 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:21.319085 2555 patch_prober.go:28] interesting pod/e2e-api-a-c3de18-openshift-default-55bf697595-2758h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 05:12:21.319514 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:21.319150 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 05:12:23.830481 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.830411 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:12:23.885392 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885358 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885392 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885401 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885637 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885435 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885637 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885468 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885637 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885483 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885637 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885523 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885637 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885574 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885624 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kdc6\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert\") pod \"ec283059-a0cf-428f-aa7c-56223b581948\" (UID: \"ec283059-a0cf-428f-aa7c-56223b581948\") " Jul 22 05:12:23.885892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885756 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:23.885892 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885860 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:23.886169 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.886154 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.886238 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.886173 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.886238 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.886197 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:12:23.886348 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.885999 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:23.886487 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.886435 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:23.887774 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.887751 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:12:23.887894 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.887873 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:23.888230 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.888202 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6" (OuterVolumeSpecName: "kube-api-access-4kdc6") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "kube-api-access-4kdc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:12:23.888230 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.888224 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec283059-a0cf-428f-aa7c-56223b581948" (UID: "ec283059-a0cf-428f-aa7c-56223b581948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:12:23.987232 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987195 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987232 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987227 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec283059-a0cf-428f-aa7c-56223b581948-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987232 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987238 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kdc6\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-kube-api-access-4kdc6\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987487 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987249 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec283059-a0cf-428f-aa7c-56223b581948-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987487 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987258 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec283059-a0cf-428f-aa7c-56223b581948-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987487 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987265 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:23.987487 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:23.987273 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec283059-a0cf-428f-aa7c-56223b581948-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:24.602179 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.602134 2555 generic.go:356] "Generic (PLEG): container finished" podID="ec283059-a0cf-428f-aa7c-56223b581948" containerID="da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f" exitCode=0 Jul 22 05:12:24.602366 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.602188 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerDied","Data":"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f"} Jul 22 05:12:24.602366 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.602197 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" Jul 22 05:12:24.602366 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.602233 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h" event={"ID":"ec283059-a0cf-428f-aa7c-56223b581948","Type":"ContainerDied","Data":"23325fccc0311626e60dcce4cc02eff976321b982cfc36a9d60d0cac9c924475"} Jul 22 05:12:24.602366 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.602256 2555 scope.go:117] "RemoveContainer" containerID="da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f" Jul 22 05:12:24.615516 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.615143 2555 scope.go:117] "RemoveContainer" containerID="0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969" Jul 22 05:12:24.624237 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.624217 2555 scope.go:117] "RemoveContainer" containerID="da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f" Jul 22 05:12:24.624528 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:12:24.624508 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f\": container with ID starting with da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f not found: ID does not exist" containerID="da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f" Jul 22 05:12:24.624605 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.624541 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f"} err="failed to get container status \"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f\": rpc error: code = NotFound desc = could not find container \"da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f\": container with ID starting with da9bb13ded889605e137159564fb37e8af0bdb7c038ee4fceaea835cce98922f not found: ID does not exist" Jul 22 05:12:24.624605 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.624568 2555 scope.go:117] "RemoveContainer" containerID="0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969" Jul 22 05:12:24.624844 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:12:24.624821 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969\": container with ID starting with 0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969 not found: ID does not exist" containerID="0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969" Jul 22 05:12:24.624890 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.624852 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969"} err="failed to get container status \"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969\": rpc error: code = NotFound desc = could not find container \"0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969\": container with ID starting with 0cb70a900c9b20ee1289a914692e904e8ce8343671116475b5b80d34a6762969 not found: ID does not exist" Jul 22 05:12:24.628565 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.628537 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:12:24.632212 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:24.632190 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-c3de18-openshift-default-55bf697595-2758h"] Jul 22 05:12:25.432133 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:25.432088 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec283059-a0cf-428f-aa7c-56223b581948" path="/var/lib/kubelet/pods/ec283059-a0cf-428f-aa7c-56223b581948/volumes" Jul 22 05:12:45.452216 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:45.452173 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:12:45.452676 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:45.452425 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" containerID="cri-o://ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544" gracePeriod=30 Jul 22 05:12:46.377281 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377246 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:12:46.377742 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377724 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.377829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377745 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.377829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377756 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.377829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377764 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.378006 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377901 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.378006 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.377914 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec283059-a0cf-428f-aa7c-56223b581948" containerName="istio-proxy" Jul 22 05:12:46.381277 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.381249 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.383735 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.383707 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c0234d-openshift-default-dockercfg-d8cw8\"" Jul 22 05:12:46.383848 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.383760 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:12:46.419926 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.419890 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hgtx\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.419926 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.419939 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420226 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.419965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420226 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420077 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420226 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420119 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420226 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420226 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420170 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420431 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420230 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.420431 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.420316 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.521608 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521571 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hgtx\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521625 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521661 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521705 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521736 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521929 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.521976 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522061 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522157 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522131 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522267 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522424 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522592 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522578 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.522956 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.522935 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.524191 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.524160 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.524423 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.524391 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.529131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.529108 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.529269 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.529195 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hgtx\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx\") pod \"e2e-shared-a-c0234d-openshift-default-747d96d459-5k957\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.693113 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.692990 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:46.845571 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.845534 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:12:46.848693 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:12:46.848663 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod354d7773_a368_45c6_a60b_93f51a44f829.slice/crio-1edb90f84cbb12e6684d75d83c91168cfb9fb265007011353d1ae6e95d2bec43 WatchSource:0}: Error finding container 1edb90f84cbb12e6684d75d83c91168cfb9fb265007011353d1ae6e95d2bec43: Status 404 returned error can't find the container with id 1edb90f84cbb12e6684d75d83c91168cfb9fb265007011353d1ae6e95d2bec43 Jul 22 05:12:46.850895 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.850862 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:46.850980 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.850932 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:46.851015 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:46.850979 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:47.728520 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:47.728478 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerStarted","Data":"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490"} Jul 22 05:12:47.728910 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:47.728541 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerStarted","Data":"1edb90f84cbb12e6684d75d83c91168cfb9fb265007011353d1ae6e95d2bec43"} Jul 22 05:12:47.747712 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:47.747644 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podStartSLOduration=1.747622888 podStartE2EDuration="1.747622888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:12:47.747379499 +0000 UTC m=+2382.795647771" watchObservedRunningTime="2026-07-22 05:12:47.747622888 +0000 UTC m=+2382.795891196" Jul 22 05:12:48.694283 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:48.694248 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:48.699285 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:48.699257 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:48.735049 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:48.734995 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:48.735980 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:48.735960 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:12:50.739735 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.739707 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:12:50.749312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.749277 2555 generic.go:356] "Generic (PLEG): container finished" podID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerID="ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544" exitCode=0 Jul 22 05:12:50.749528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.749348 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" Jul 22 05:12:50.749528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.749374 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerDied","Data":"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544"} Jul 22 05:12:50.749528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.749456 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75" event={"ID":"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53","Type":"ContainerDied","Data":"4ddaae4548a3fca30edc41d8464160141ac9a43e309ec4d87ab96e46f44d0731"} Jul 22 05:12:50.749528 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.749482 2555 scope.go:117] "RemoveContainer" containerID="ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544" Jul 22 05:12:50.759685 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.759654 2555 scope.go:117] "RemoveContainer" containerID="c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc" Jul 22 05:12:50.763237 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763192 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763372 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763284 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763372 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763351 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763518 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763495 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763576 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763539 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763641 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763594 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhmtj\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763641 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763635 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763752 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763665 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.763752 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.763695 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token\") pod \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\" (UID: \"05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53\") " Jul 22 05:12:50.764171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764048 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:50.764171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764003 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:50.764171 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764078 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:50.764412 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764245 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.764412 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764268 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.764412 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764283 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.764805 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.764776 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:12:50.765071 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.765042 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data" (OuterVolumeSpecName: "istio-data") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:50.766990 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.766952 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj" (OuterVolumeSpecName: "kube-api-access-fhmtj") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "kube-api-access-fhmtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:12:50.767368 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.767342 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token" (OuterVolumeSpecName: "istio-token") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:12:50.767824 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.767803 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:12:50.768053 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.768017 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" (UID: "05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:12:50.772014 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.771996 2555 scope.go:117] "RemoveContainer" containerID="ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544" Jul 22 05:12:50.772442 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:12:50.772423 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544\": container with ID starting with ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544 not found: ID does not exist" containerID="ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544" Jul 22 05:12:50.772515 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.772452 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544"} err="failed to get container status \"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544\": rpc error: code = NotFound desc = could not find container \"ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544\": container with ID starting with ec2ddb919714873052a01d8d5f404fada8f12d960915a8ecdc1bce444fd04544 not found: ID does not exist" Jul 22 05:12:50.772515 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.772473 2555 scope.go:117] "RemoveContainer" containerID="c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc" Jul 22 05:12:50.772735 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:12:50.772713 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc\": container with ID starting with c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc not found: ID does not exist" containerID="c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc" Jul 22 05:12:50.772791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.772745 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc"} err="failed to get container status \"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc\": rpc error: code = NotFound desc = could not find container \"c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc\": container with ID starting with c137470a23364d1f9070661a4b291f7376d8deee81d74805173193c7961effcc not found: ID does not exist" Jul 22 05:12:50.865620 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865575 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.865620 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865611 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.865620 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865626 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhmtj\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-kube-api-access-fhmtj\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.865878 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865640 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.865878 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865651 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:50.865878 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:50.865661 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:12:51.072244 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:51.072200 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:12:51.076288 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:51.076254 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-04dfa1-openshift-default-68977f8d8f-2rd75"] Jul 22 05:12:51.432766 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:51.432671 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" path="/var/lib/kubelet/pods/05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53/volumes" Jul 22 05:12:54.933227 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933188 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:12:54.933827 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933810 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.933872 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933830 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.933872 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933842 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.933872 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933851 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.933998 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.933986 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.934063 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.934004 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="05c9c726-ddb8-4d6e-9ad0-55bd0ed48b53" containerName="istio-proxy" Jul 22 05:12:54.944337 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.944302 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:54.946935 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.946910 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1a3bbb-openshift-default-dockercfg-4wn6r\"" Jul 22 05:12:54.954530 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:54.954503 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:12:55.005817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005763 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.005817 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005808 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005835 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005893 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005925 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005965 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvclg\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.005986 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.006115 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.006059 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107574 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107532 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107778 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107588 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107778 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107620 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107778 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107654 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107778 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107684 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvclg\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.107778 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.107718 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108104 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108054 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108104 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108080 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108104 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108085 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108244 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108152 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108305 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108277 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108393 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108328 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108662 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108639 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.108971 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.108945 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.110468 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.110439 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.110673 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.110654 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.115388 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.115365 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.115526 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.115508 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvclg\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg\") pod \"e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.257879 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.257847 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:55.441316 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.441288 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:12:55.443605 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:12:55.443559 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabb1f90d_6109_4e5f_826b_c5b064409a25.slice/crio-f6939c9b2d66da525377f4cb721d8d16fadfbad6da6d0340bf37b091862e9c8e WatchSource:0}: Error finding container f6939c9b2d66da525377f4cb721d8d16fadfbad6da6d0340bf37b091862e9c8e: Status 404 returned error can't find the container with id f6939c9b2d66da525377f4cb721d8d16fadfbad6da6d0340bf37b091862e9c8e Jul 22 05:12:55.446132 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.446075 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:55.446282 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.446255 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:55.446358 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.446340 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:12:55.780829 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.780793 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerStarted","Data":"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff"} Jul 22 05:12:55.781081 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.780846 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerStarted","Data":"f6939c9b2d66da525377f4cb721d8d16fadfbad6da6d0340bf37b091862e9c8e"} Jul 22 05:12:55.800392 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:55.800328 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podStartSLOduration=1.800312377 podStartE2EDuration="1.800312377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:12:55.798635465 +0000 UTC m=+2390.846903736" watchObservedRunningTime="2026-07-22 05:12:55.800312377 +0000 UTC m=+2390.848580638" Jul 22 05:12:56.258904 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:56.258859 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:56.260467 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:56.260439 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 05:12:56.260589 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:56.260491 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 05:12:57.263100 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:57.263068 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:57.791529 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:57.791497 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:12:57.792565 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:12:57.792541 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:13:04.777184 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:04.777144 2555 patch_prober.go:28] interesting pod/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 05:13:04.777564 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:04.777266 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 05:13:05.552314 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.552284 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:13:05.560511 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.560486 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:13:05.843276 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.843240 2555 generic.go:356] "Generic (PLEG): container finished" podID="354d7773-a368-45c6-a60b-93f51a44f829" containerID="bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490" exitCode=0 Jul 22 05:13:05.843687 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.843316 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerDied","Data":"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490"} Jul 22 05:13:05.843813 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.843740 2555 scope.go:117] "RemoveContainer" containerID="bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490" Jul 22 05:13:05.844832 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.844805 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:05.844910 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.844893 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:05.844951 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:05.844937 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:06.499119 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:06.499092 2555 scope.go:117] "RemoveContainer" containerID="a82f4e35bc419933a600fd0ea49fe4b66211ed2b4ca1ae2128c332aa2d03dcd0" Jul 22 05:13:06.851618 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:06.851510 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerStarted","Data":"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711"} Jul 22 05:13:07.693318 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:07.693272 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:13:07.693992 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:07.693963 2555 patch_prober.go:28] interesting pod/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 05:13:07.694120 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:07.694044 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 05:13:08.693825 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:08.693784 2555 patch_prober.go:28] interesting pod/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 05:13:08.694255 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:08.693840 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 05:13:09.693551 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:09.693518 2555 patch_prober.go:28] interesting pod/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 05:13:09.693788 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:09.693766 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 05:13:10.693971 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:10.693934 2555 patch_prober.go:28] interesting pod/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 22 05:13:10.694517 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:10.694479 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 22 05:13:11.697394 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:11.697309 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:13:11.697806 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:11.697784 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:13:11.698699 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:11.698679 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:13:13.832199 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:13.832157 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 05:13:13.832600 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:13.832228 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 05:13:14.908842 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.908803 2555 generic.go:356] "Generic (PLEG): container finished" podID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerID="5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff" exitCode=0 Jul 22 05:13:14.909409 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.908885 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerDied","Data":"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff"} Jul 22 05:13:14.909497 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.909465 2555 scope.go:117] "RemoveContainer" containerID="5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff" Jul 22 05:13:14.910783 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.910751 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:14.910878 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.910862 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:14.910919 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:14.910908 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 05:13:15.924483 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:15.924442 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerStarted","Data":"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915"} Jul 22 05:13:16.258893 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:16.258858 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:13:16.259131 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:16.259103 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 05:13:16.259197 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:16.259162 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 05:13:17.258914 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:17.258872 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 05:13:17.259364 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:17.258931 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 05:13:18.258330 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:18.258287 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 05:13:18.258529 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:18.258359 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 05:13:19.259111 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:19.259069 2555 patch_prober.go:28] interesting pod/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Jul 22 05:13:19.259540 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:19.259129 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Jul 22 05:13:20.262189 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:20.262153 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:13:20.262653 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:20.262564 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:13:20.263388 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:13:20.263368 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:18:05.588057 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:05.587936 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:18:05.596937 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:05.596906 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5752d_52634ac6-7c8f-4615-8a35-33994da99ed7/ovn-acl-logging/0.log" Jul 22 05:18:57.759139 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.759098 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7614-139631815387824"] Jul 22 05:18:57.762726 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.762708 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:18:57.765108 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.765083 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pz2cr\"" Jul 22 05:18:57.770134 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.770091 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7614-139631815387824"] Jul 22 05:18:57.829370 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.829340 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84jr\" (UniqueName: \"kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr\") pod \"test-curl-7614-139631815387824\" (UID: \"c40b14ec-686f-4a98-9d9e-9077188197ae\") " pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:18:57.930631 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.930591 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84jr\" (UniqueName: \"kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr\") pod \"test-curl-7614-139631815387824\" (UID: \"c40b14ec-686f-4a98-9d9e-9077188197ae\") " pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:18:57.938856 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:57.938824 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84jr\" (UniqueName: \"kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr\") pod \"test-curl-7614-139631815387824\" (UID: \"c40b14ec-686f-4a98-9d9e-9077188197ae\") " pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:18:58.075008 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:58.074926 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:18:58.232607 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:58.232572 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7614-139631815387824"] Jul 22 05:18:58.235708 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:18:58.235660 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc40b14ec_686f_4a98_9d9e_9077188197ae.slice/crio-76a3c40b28c8126f23e39727ea61fe22d2161f9082ac78f7754083f80af8d9d2 WatchSource:0}: Error finding container 76a3c40b28c8126f23e39727ea61fe22d2161f9082ac78f7754083f80af8d9d2: Status 404 returned error can't find the container with id 76a3c40b28c8126f23e39727ea61fe22d2161f9082ac78f7754083f80af8d9d2 Jul 22 05:18:58.237251 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:58.237225 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 05:18:58.851046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:58.850990 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7614-139631815387824" event={"ID":"c40b14ec-686f-4a98-9d9e-9077188197ae","Type":"ContainerStarted","Data":"76a3c40b28c8126f23e39727ea61fe22d2161f9082ac78f7754083f80af8d9d2"} Jul 22 05:18:59.858750 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:59.858710 2555 generic.go:356] "Generic (PLEG): container finished" podID="c40b14ec-686f-4a98-9d9e-9077188197ae" containerID="c3ad6fe432b4b110846e0746f5c292af699fec6556458ea9f71c57500a36dbd1" exitCode=0 Jul 22 05:18:59.859278 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:18:59.858799 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7614-139631815387824" event={"ID":"c40b14ec-686f-4a98-9d9e-9077188197ae","Type":"ContainerDied","Data":"c3ad6fe432b4b110846e0746f5c292af699fec6556458ea9f71c57500a36dbd1"} Jul 22 05:19:01.027992 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.027949 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:19:01.060368 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.060333 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b84jr\" (UniqueName: \"kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr\") pod \"c40b14ec-686f-4a98-9d9e-9077188197ae\" (UID: \"c40b14ec-686f-4a98-9d9e-9077188197ae\") " Jul 22 05:19:01.061123 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.061098 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7614-139631815387824_c40b14ec-686f-4a98-9d9e-9077188197ae/test-curl-7614-139631815387824/0.log" Jul 22 05:19:01.062598 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.062575 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr" (OuterVolumeSpecName: "kube-api-access-b84jr") pod "c40b14ec-686f-4a98-9d9e-9077188197ae" (UID: "c40b14ec-686f-4a98-9d9e-9077188197ae"). InnerVolumeSpecName "kube-api-access-b84jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:19:01.084361 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.084306 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7614-139631815387824"] Jul 22 05:19:01.087477 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.087442 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7614-139631815387824"] Jul 22 05:19:01.161822 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.161693 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b84jr\" (UniqueName: \"kubernetes.io/projected/c40b14ec-686f-4a98-9d9e-9077188197ae-kube-api-access-b84jr\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:01.432227 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.432129 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c40b14ec-686f-4a98-9d9e-9077188197ae" path="/var/lib/kubelet/pods/c40b14ec-686f-4a98-9d9e-9077188197ae/volumes" Jul 22 05:19:01.871407 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.871376 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815387824" Jul 22 05:19:01.871407 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:01.871377 2555 scope.go:117] "RemoveContainer" containerID="c3ad6fe432b4b110846e0746f5c292af699fec6556458ea9f71c57500a36dbd1" Jul 22 05:19:03.774730 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.774689 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7614-139631815377952"] Jul 22 05:19:03.775243 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.775224 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c40b14ec-686f-4a98-9d9e-9077188197ae" containerName="test-curl-7614-139631815387824" Jul 22 05:19:03.775294 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.775244 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="c40b14ec-686f-4a98-9d9e-9077188197ae" containerName="test-curl-7614-139631815387824" Jul 22 05:19:03.775327 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.775310 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="c40b14ec-686f-4a98-9d9e-9077188197ae" containerName="test-curl-7614-139631815387824" Jul 22 05:19:03.780053 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.780006 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:03.782631 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.782603 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pz2cr\"" Jul 22 05:19:03.783949 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.783924 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7614-139631815377952"] Jul 22 05:19:03.889883 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.889828 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdmr\" (UniqueName: \"kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr\") pod \"test-curl-7614-139631815377952\" (UID: \"51ec8aca-684f-48a9-b7b2-bdaf60a3d129\") " pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:03.991362 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.991303 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdmr\" (UniqueName: \"kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr\") pod \"test-curl-7614-139631815377952\" (UID: \"51ec8aca-684f-48a9-b7b2-bdaf60a3d129\") " pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:03.999952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:03.999922 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdmr\" (UniqueName: \"kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr\") pod \"test-curl-7614-139631815377952\" (UID: \"51ec8aca-684f-48a9-b7b2-bdaf60a3d129\") " pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:04.091646 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:04.091477 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:04.245285 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:04.245070 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7614-139631815377952"] Jul 22 05:19:04.248062 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:19:04.248009 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51ec8aca_684f_48a9_b7b2_bdaf60a3d129.slice/crio-593ccbfcb24c650c4c8015d5afc3eb660eb3eb057256c7d3b059974c9976101b WatchSource:0}: Error finding container 593ccbfcb24c650c4c8015d5afc3eb660eb3eb057256c7d3b059974c9976101b: Status 404 returned error can't find the container with id 593ccbfcb24c650c4c8015d5afc3eb660eb3eb057256c7d3b059974c9976101b Jul 22 05:19:04.891768 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:04.891674 2555 generic.go:356] "Generic (PLEG): container finished" podID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" containerID="3468214a2a30d73a63451b403f9d2eeba12f2f69b3a8a0976193660c7339992c" exitCode=0 Jul 22 05:19:04.891768 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:04.891737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7614-139631815377952" event={"ID":"51ec8aca-684f-48a9-b7b2-bdaf60a3d129","Type":"ContainerDied","Data":"3468214a2a30d73a63451b403f9d2eeba12f2f69b3a8a0976193660c7339992c"} Jul 22 05:19:04.892213 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:04.891770 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7614-139631815377952" event={"ID":"51ec8aca-684f-48a9-b7b2-bdaf60a3d129","Type":"ContainerStarted","Data":"593ccbfcb24c650c4c8015d5afc3eb660eb3eb057256c7d3b059974c9976101b"} Jul 22 05:19:06.049312 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.049285 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:06.067991 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.067955 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7614-139631815377952_51ec8aca-684f-48a9-b7b2-bdaf60a3d129/test-curl-7614-139631815377952/0.log" Jul 22 05:19:06.089182 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.089150 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7614-139631815377952"] Jul 22 05:19:06.094067 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.094040 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7614-139631815377952"] Jul 22 05:19:06.113189 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.113156 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrdmr\" (UniqueName: \"kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr\") pod \"51ec8aca-684f-48a9-b7b2-bdaf60a3d129\" (UID: \"51ec8aca-684f-48a9-b7b2-bdaf60a3d129\") " Jul 22 05:19:06.115384 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.115352 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr" (OuterVolumeSpecName: "kube-api-access-rrdmr") pod "51ec8aca-684f-48a9-b7b2-bdaf60a3d129" (UID: "51ec8aca-684f-48a9-b7b2-bdaf60a3d129"). InnerVolumeSpecName "kube-api-access-rrdmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:19:06.214971 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.214880 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrdmr\" (UniqueName: \"kubernetes.io/projected/51ec8aca-684f-48a9-b7b2-bdaf60a3d129-kube-api-access-rrdmr\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:06.904543 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.904375 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:06.904543 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:06.904394 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="593ccbfcb24c650c4c8015d5afc3eb660eb3eb057256c7d3b059974c9976101b" Jul 22 05:19:06.918986 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:19:06.918937 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7614-139631815377952\" is forbidden: User \"system:node:ip-10-0-133-75.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-75.ec2.internal' and this object" podUID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" pod="opendatahub/test-curl-7614-139631815377952" Jul 22 05:19:07.431775 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:07.431739 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" path="/var/lib/kubelet/pods/51ec8aca-684f-48a9-b7b2-bdaf60a3d129/volumes" Jul 22 05:19:33.549285 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:33.549175 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:19:33.549932 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:33.549534 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" containerID="cri-o://29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711" gracePeriod=30 Jul 22 05:19:38.813777 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.813671 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:19:38.852395 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852341 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hgtx\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.852395 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852395 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.852718 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852444 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.852718 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852550 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.852718 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852626 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.852718 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852658 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852788 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852784 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852833 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852879 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data\") pod \"354d7773-a368-45c6-a60b-93f51a44f829\" (UID: \"354d7773-a368-45c6-a60b-93f51a44f829\") " Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.852909 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853004 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853294 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853428 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853448 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/354d7773-a368-45c6-a60b-93f51a44f829-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853463 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.853719 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853477 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.854179 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.853995 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data" (OuterVolumeSpecName: "istio-data") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:19:38.855360 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.855329 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx" (OuterVolumeSpecName: "kube-api-access-7hgtx") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "kube-api-access-7hgtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:19:38.855619 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.855583 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token" (OuterVolumeSpecName: "istio-token") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:19:38.855697 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.855597 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:19:38.855697 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.855667 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "354d7773-a368-45c6-a60b-93f51a44f829" (UID: "354d7773-a368-45c6-a60b-93f51a44f829"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:19:38.954776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.954717 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.954776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.954765 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.954776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.954774 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/354d7773-a368-45c6-a60b-93f51a44f829-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.954776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.954785 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hgtx\" (UniqueName: \"kubernetes.io/projected/354d7773-a368-45c6-a60b-93f51a44f829-kube-api-access-7hgtx\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:38.954776 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:38.954796 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/354d7773-a368-45c6-a60b-93f51a44f829-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:19:39.092469 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.092352 2555 generic.go:356] "Generic (PLEG): container finished" podID="354d7773-a368-45c6-a60b-93f51a44f829" containerID="29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711" exitCode=0 Jul 22 05:19:39.092469 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.092419 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerDied","Data":"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711"} Jul 22 05:19:39.092469 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.092442 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" Jul 22 05:19:39.092763 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.092489 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957" event={"ID":"354d7773-a368-45c6-a60b-93f51a44f829","Type":"ContainerDied","Data":"1edb90f84cbb12e6684d75d83c91168cfb9fb265007011353d1ae6e95d2bec43"} Jul 22 05:19:39.092763 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.092515 2555 scope.go:117] "RemoveContainer" containerID="29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711" Jul 22 05:19:39.103958 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.103882 2555 scope.go:117] "RemoveContainer" containerID="bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490" Jul 22 05:19:39.113497 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.113475 2555 scope.go:117] "RemoveContainer" containerID="29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711" Jul 22 05:19:39.113784 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:19:39.113758 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711\": container with ID starting with 29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711 not found: ID does not exist" containerID="29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711" Jul 22 05:19:39.113833 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.113802 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711"} err="failed to get container status \"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711\": rpc error: code = NotFound desc = could not find container \"29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711\": container with ID starting with 29297e472afce977a05321799e9eae640cc40268c43d4b8a5c4d6128e7cf9711 not found: ID does not exist" Jul 22 05:19:39.113885 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.113832 2555 scope.go:117] "RemoveContainer" containerID="bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490" Jul 22 05:19:39.114136 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:19:39.114117 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490\": container with ID starting with bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490 not found: ID does not exist" containerID="bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490" Jul 22 05:19:39.114194 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.114143 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490"} err="failed to get container status \"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490\": rpc error: code = NotFound desc = could not find container \"bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490\": container with ID starting with bcf7e002e80f8527dca13969050db64979850b15f71425be976f6b708750c490 not found: ID does not exist" Jul 22 05:19:39.115071 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.115046 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:19:39.120866 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.120838 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c0234d-openshift-default-747d96d459-5k957"] Jul 22 05:19:39.431420 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:39.431316 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354d7773-a368-45c6-a60b-93f51a44f829" path="/var/lib/kubelet/pods/354d7773-a368-45c6-a60b-93f51a44f829/volumes" Jul 22 05:19:55.352282 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:55.352238 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:19:55.352826 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:55.352592 2555 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" containerID="cri-o://ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915" gracePeriod=30 Jul 22 05:19:56.068088 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:19:56.068048 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff5b95d65-bkbxh_2c440fc8-81c9-4d93-9958-e5684bba7db4/authorino/0.log" Jul 22 05:20:00.605121 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.605078 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:20:00.685485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685343 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685398 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685428 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvclg\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685459 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685485 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685484 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685516 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685585 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685682 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685718 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert\") pod \"abb1f90d-6109-4e5f-826b-c5b064409a25\" (UID: \"abb1f90d-6109-4e5f-826b-c5b064409a25\") " Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685902 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:20:00.685952 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.685917 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:20:00.686247 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686017 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:20:00.686327 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686301 2555 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.686327 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686321 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:20:00.686439 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686331 2555 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-workload-certs\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.686439 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686346 2555 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-credential-socket\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.686868 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.686841 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data" (OuterVolumeSpecName: "istio-data") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:20:00.688344 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.688318 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token" (OuterVolumeSpecName: "istio-token") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:20:00.688446 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.688340 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg" (OuterVolumeSpecName: "kube-api-access-tvclg") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "kube-api-access-tvclg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:20:00.688489 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.688450 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 05:20:00.688623 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.688600 2555 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "abb1f90d-6109-4e5f-826b-c5b064409a25" (UID: "abb1f90d-6109-4e5f-826b-c5b064409a25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 05:20:00.787219 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787182 2555 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-token\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.787219 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787212 2555 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/abb1f90d-6109-4e5f-826b-c5b064409a25-istiod-ca-cert\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.787219 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787221 2555 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-envoy\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.787219 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787230 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvclg\" (UniqueName: \"kubernetes.io/projected/abb1f90d-6109-4e5f-826b-c5b064409a25-kube-api-access-tvclg\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.787491 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787239 2555 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-data\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:00.787491 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:00.787248 2555 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/abb1f90d-6109-4e5f-826b-c5b064409a25-istio-podinfo\") on node \"ip-10-0-133-75.ec2.internal\" DevicePath \"\"" Jul 22 05:20:01.220630 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.220593 2555 generic.go:356] "Generic (PLEG): container finished" podID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerID="ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915" exitCode=0 Jul 22 05:20:01.220863 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.220666 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" Jul 22 05:20:01.220863 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.220683 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerDied","Data":"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915"} Jul 22 05:20:01.220863 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.220765 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm" event={"ID":"abb1f90d-6109-4e5f-826b-c5b064409a25","Type":"ContainerDied","Data":"f6939c9b2d66da525377f4cb721d8d16fadfbad6da6d0340bf37b091862e9c8e"} Jul 22 05:20:01.220863 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.220803 2555 scope.go:117] "RemoveContainer" containerID="ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915" Jul 22 05:20:01.231786 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.231762 2555 scope.go:117] "RemoveContainer" containerID="5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff" Jul 22 05:20:01.240967 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.240943 2555 scope.go:117] "RemoveContainer" containerID="ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915" Jul 22 05:20:01.241392 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:20:01.241356 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915\": container with ID starting with ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915 not found: ID does not exist" containerID="ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915" Jul 22 05:20:01.241488 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.241404 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915"} err="failed to get container status \"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915\": rpc error: code = NotFound desc = could not find container \"ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915\": container with ID starting with ad79f87c469eab710c151118c19a54758041c8c11921c6367ee7ac34df033915 not found: ID does not exist" Jul 22 05:20:01.241488 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.241428 2555 scope.go:117] "RemoveContainer" containerID="5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff" Jul 22 05:20:01.241728 ip-10-0-133-75 kubenswrapper[2555]: E0722 05:20:01.241710 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff\": container with ID starting with 5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff not found: ID does not exist" containerID="5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff" Jul 22 05:20:01.241774 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.241741 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff"} err="failed to get container status \"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff\": rpc error: code = NotFound desc = could not find container \"5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff\": container with ID starting with 5c50607de50883a517c1d68d7f68d649175d2053d11119836272730b9cadccff not found: ID does not exist" Jul 22 05:20:01.243926 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.243900 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:20:01.248772 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.248746 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1a3bbb-openshift-default-85dfcb4959-8ltnm"] Jul 22 05:20:01.431491 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.431446 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" path="/var/lib/kubelet/pods/abb1f90d-6109-4e5f-826b-c5b064409a25/volumes" Jul 22 05:20:01.567475 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.567446 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wxltv_241e0eae-d237-46f1-9151-c9302d843c75/manager/0.log" Jul 22 05:20:01.696407 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.696376 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bfl5_b81bea95-d97a-4049-9f97-c79f21dbf0f3/manager/0.log" Jul 22 05:20:01.823855 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.823758 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865d8b7674-wczgw_00019a52-7e92-4e78-9628-788953f92677/manager/0.log" Jul 22 05:20:01.936640 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:01.936610 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r7cm5_2bc3a940-3aa9-429c-b1c1-ad00677d8f53/server/0.log" Jul 22 05:20:02.058313 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:02.058274 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dqv4n_b89822bc-24c2-4c21-b858-0d592985075f/manager/0.log" Jul 22 05:20:02.434192 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:02.434159 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5567574457-s8tcd_31944654-99d5-4b4b-9b2a-fe3b7db42e29/copy-manifests/0.log" Jul 22 05:20:02.441229 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:02.441198 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5567574457-s8tcd_31944654-99d5-4b4b-9b2a-fe3b7db42e29/manager/0.log" Jul 22 05:20:04.576297 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:04.576256 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff5b95d65-bkbxh_2c440fc8-81c9-4d93-9958-e5684bba7db4/authorino/0.log" Jul 22 05:20:04.700739 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:04.700707 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z2sl6_8eff9a5e-1131-4be2-bb35-067dfda38a46/manager/0.log" Jul 22 05:20:04.814677 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:04.814644 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lmdvn_ca4c5a77-c948-45f0-ad44-da876ea54fbd/manager/0.log" Jul 22 05:20:05.067427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:05.067384 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r55h8_6aa3220a-7609-422d-90f9-3e94369fde6e/registry-server/0.log" Jul 22 05:20:05.200201 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:05.200160 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7gdpt_6a106dee-2e29-4910-977d-e4288c6502f1/manager/0.log" Jul 22 05:20:05.443230 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:05.443134 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8krms_4478a13d-7890-45a5-b1fc-2e6e975380a3/manager/0.log" Jul 22 05:20:05.831724 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:05.831684 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678j5fmj_03259dd4-d21f-454a-85c5-b2ea70adfa79/istio-proxy/0.log" Jul 22 05:20:06.191359 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:06.191269 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d644bdf8d-qgsln_a7476468-64fe-4598-ba2b-e5775a458b0d/kube-auth-proxy/0.log" Jul 22 05:20:06.324457 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:06.324420 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/1.log" Jul 22 05:20:07.162971 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:07.162935 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-xh6js_959749be-0866-43ed-af76-47966b9d5428/main/0.log" Jul 22 05:20:07.527852 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:07.527812 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-njq9q_4a75524c-f539-4ffa-b3a7-507c49e2c277/main/0.log" Jul 22 05:20:13.480018 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.479980 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84vmg/must-gather-dlj6j"] Jul 22 05:20:13.480466 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480449 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.480466 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480468 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480478 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480483 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480492 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480498 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480504 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" containerName="test-curl-7614-139631815377952" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480511 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" containerName="test-curl-7614-139631815377952" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480544 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480549 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480550 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480625 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="51ec8aca-684f-48a9-b7b2-bdaf60a3d129" containerName="test-curl-7614-139631815377952" Jul 22 05:20:13.480791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480635 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480642 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="abb1f90d-6109-4e5f-826b-c5b064409a25" containerName="istio-proxy" Jul 22 05:20:13.480791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480649 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.480791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.480656 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="354d7773-a368-45c6-a60b-93f51a44f829" containerName="istio-proxy" Jul 22 05:20:13.485802 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.485783 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.488871 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.488839 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84vmg\"/\"openshift-service-ca.crt\"" Jul 22 05:20:13.489212 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.489184 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-84vmg\"/\"default-dockercfg-vt96m\"" Jul 22 05:20:13.489972 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.489950 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-84vmg\"/\"kube-root-ca.crt\"" Jul 22 05:20:13.492712 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.492683 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84vmg/must-gather-dlj6j"] Jul 22 05:20:13.619421 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.619381 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgh5h\" (UniqueName: \"kubernetes.io/projected/69451b78-ecae-410e-a88c-d95d9c2c3cda-kube-api-access-zgh5h\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.619421 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.619426 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/69451b78-ecae-410e-a88c-d95d9c2c3cda-must-gather-output\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.721046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.720985 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgh5h\" (UniqueName: \"kubernetes.io/projected/69451b78-ecae-410e-a88c-d95d9c2c3cda-kube-api-access-zgh5h\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.721220 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.721073 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/69451b78-ecae-410e-a88c-d95d9c2c3cda-must-gather-output\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.721427 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.721406 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/69451b78-ecae-410e-a88c-d95d9c2c3cda-must-gather-output\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.728732 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.728704 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgh5h\" (UniqueName: \"kubernetes.io/projected/69451b78-ecae-410e-a88c-d95d9c2c3cda-kube-api-access-zgh5h\") pod \"must-gather-dlj6j\" (UID: \"69451b78-ecae-410e-a88c-d95d9c2c3cda\") " pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.797491 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.797458 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84vmg/must-gather-dlj6j" Jul 22 05:20:13.943578 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:13.943549 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84vmg/must-gather-dlj6j"] Jul 22 05:20:13.946055 ip-10-0-133-75 kubenswrapper[2555]: W0722 05:20:13.946002 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69451b78_ecae_410e_a88c_d95d9c2c3cda.slice/crio-20792f6f3be5876bde2634c2ed9bc3e3283528a04a3a88cd1f275a601e63370a WatchSource:0}: Error finding container 20792f6f3be5876bde2634c2ed9bc3e3283528a04a3a88cd1f275a601e63370a: Status 404 returned error can't find the container with id 20792f6f3be5876bde2634c2ed9bc3e3283528a04a3a88cd1f275a601e63370a Jul 22 05:20:14.295788 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:14.295727 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84vmg/must-gather-dlj6j" event={"ID":"69451b78-ecae-410e-a88c-d95d9c2c3cda","Type":"ContainerStarted","Data":"20792f6f3be5876bde2634c2ed9bc3e3283528a04a3a88cd1f275a601e63370a"} Jul 22 05:20:15.305471 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:15.305425 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84vmg/must-gather-dlj6j" event={"ID":"69451b78-ecae-410e-a88c-d95d9c2c3cda","Type":"ContainerStarted","Data":"d9f5c0e32c30bfd40a154a8223a729e28ceb25a4671a35eb97cc8cde281024e0"} Jul 22 05:20:15.305957 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:15.305485 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84vmg/must-gather-dlj6j" event={"ID":"69451b78-ecae-410e-a88c-d95d9c2c3cda","Type":"ContainerStarted","Data":"3a0472bad0a9950f193913242bf15d7597cdf11cd9fd4a62bce7914f5be32f64"} Jul 22 05:20:15.322800 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:15.322721 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84vmg/must-gather-dlj6j" podStartSLOduration=1.419827023 podStartE2EDuration="2.322696249s" totalImagesPullingTime="902.869226ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:20:15.321003401 +0000 UTC m=+2830.369271660" watchObservedRunningTime="2026-07-22 05:20:15.322696249 +0000 UTC m=+2830.370964510" Jul 22 05:20:16.499563 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:16.499525 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-95d2d_a3b3084d-a1cd-481b-abd1-c4cd3d9dd3c8/global-pull-secret-syncer/0.log" Jul 22 05:20:16.700744 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:16.700685 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wpln7_31aaf12f-2127-4cb9-8d65-959776536a9e/konnectivity-agent/0.log" Jul 22 05:20:16.722046 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:16.721988 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-75.ec2.internal_8a511dbb64386472f7853b93500b3424/haproxy/0.log" Jul 22 05:20:22.375053 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.373063 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5ff5b95d65-bkbxh_2c440fc8-81c9-4d93-9958-e5684bba7db4/authorino/0.log" Jul 22 05:20:22.410051 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.409515 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z2sl6_8eff9a5e-1131-4be2-bb35-067dfda38a46/manager/0.log" Jul 22 05:20:22.437057 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.436948 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lmdvn_ca4c5a77-c948-45f0-ad44-da876ea54fbd/manager/0.log" Jul 22 05:20:22.522812 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.522767 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-r55h8_6aa3220a-7609-422d-90f9-3e94369fde6e/registry-server/0.log" Jul 22 05:20:22.800288 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.800252 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-7gdpt_6a106dee-2e29-4910-977d-e4288c6502f1/manager/0.log" Jul 22 05:20:22.941767 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:22.939314 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8krms_4478a13d-7890-45a5-b1fc-2e6e975380a3/manager/0.log" Jul 22 05:20:24.697654 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.697620 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vv7b8_ad1b7570-747d-45bd-aaf8-ebc12c73c426/kube-state-metrics/0.log" Jul 22 05:20:24.735258 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.735191 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vv7b8_ad1b7570-747d-45bd-aaf8-ebc12c73c426/kube-rbac-proxy-main/0.log" Jul 22 05:20:24.781557 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.781525 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-vv7b8_ad1b7570-747d-45bd-aaf8-ebc12c73c426/kube-rbac-proxy-self/0.log" Jul 22 05:20:24.849117 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.849090 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-hk2tj_290a7e21-63fb-49e8-8a98-f4bb9dcc0bd2/monitoring-plugin/0.log" Jul 22 05:20:24.965603 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.965463 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fw8ht_c612385a-5af9-4b1d-af28-e5e6ee949c96/node-exporter/0.log" Jul 22 05:20:24.987281 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:24.987248 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fw8ht_c612385a-5af9-4b1d-af28-e5e6ee949c96/kube-rbac-proxy/0.log" Jul 22 05:20:25.009690 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.009648 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fw8ht_c612385a-5af9-4b1d-af28-e5e6ee949c96/init-textfile/0.log" Jul 22 05:20:25.438392 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.438348 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-jhtkm_b22eef00-8b14-4fdc-a5a9-e2f86d4a03e4/prometheus-operator-admission-webhook/0.log" Jul 22 05:20:25.476259 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.476213 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696fb57c6b-lkmgb_0a62ecf7-3880-458b-a3ef-66543d66bd0f/telemeter-client/0.log" Jul 22 05:20:25.498227 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.498197 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696fb57c6b-lkmgb_0a62ecf7-3880-458b-a3ef-66543d66bd0f/reload/0.log" Jul 22 05:20:25.521791 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.521758 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-696fb57c6b-lkmgb_0a62ecf7-3880-458b-a3ef-66543d66bd0f/kube-rbac-proxy/0.log" Jul 22 05:20:25.559182 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.559138 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/thanos-query/0.log" Jul 22 05:20:25.581663 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.581593 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/kube-rbac-proxy-web/0.log" Jul 22 05:20:25.602069 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.601998 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/kube-rbac-proxy/0.log" Jul 22 05:20:25.621725 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.621688 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/prom-label-proxy/0.log" Jul 22 05:20:25.642189 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.642150 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/kube-rbac-proxy-rules/0.log" Jul 22 05:20:25.662808 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:25.662773 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5cf98d6887-f8bfj_7dad0cf6-a79a-44a1-97e4-f7c0213b66dd/kube-rbac-proxy-metrics/0.log" Jul 22 05:20:27.666576 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.666525 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd"] Jul 22 05:20:27.675520 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.675479 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.678206 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.678173 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd"] Jul 22 05:20:27.715997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.715244 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-sys\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.715997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.715322 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-lib-modules\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.715997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.715405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-proc\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.715997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.715517 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88tkn\" (UniqueName: \"kubernetes.io/projected/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-kube-api-access-88tkn\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.715997 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.715581 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-podres\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816543 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-lib-modules\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816742 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816570 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-proc\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816742 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816656 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88tkn\" (UniqueName: \"kubernetes.io/projected/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-kube-api-access-88tkn\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816742 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816705 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-podres\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816906 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816772 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-sys\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.816906 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.816887 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-sys\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.817061 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.817041 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-lib-modules\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.817145 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.817129 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-podres\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.817199 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.817185 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-proc\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.826064 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.826009 2555 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88tkn\" (UniqueName: \"kubernetes.io/projected/0b0ce18e-3188-4b7b-995c-49fc25b70aa0-kube-api-access-88tkn\") pod \"perf-node-gather-daemonset-xxdnd\" (UID: \"0b0ce18e-3188-4b7b-995c-49fc25b70aa0\") " pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:27.990914 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:27.990813 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:28.208256 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.207331 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd"] Jul 22 05:20:28.270274 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.270208 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dddc9f556-wqscb_63d326f7-ae10-4dc0-afec-3c4e2e398525/console/0.log" Jul 22 05:20:28.313225 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.313077 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-sxgrg_c46c07cc-63c3-4276-8f63-6e142f164fa7/download-server/0.log" Jul 22 05:20:28.438848 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.438794 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" event={"ID":"0b0ce18e-3188-4b7b-995c-49fc25b70aa0","Type":"ContainerStarted","Data":"b44f9795e1c8979acfb650c6f62d3591dc2140b796df4a7db04752c1e11862eb"} Jul 22 05:20:28.439081 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.438876 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" event={"ID":"0b0ce18e-3188-4b7b-995c-49fc25b70aa0","Type":"ContainerStarted","Data":"06b4c13df818711e908d2c15104a1e3105c3a148a022a9b84728edc189321811"} Jul 22 05:20:28.439158 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.439079 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:28.457256 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:28.457175 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" podStartSLOduration=1.45714939 podStartE2EDuration="1.45714939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 05:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:20:28.453962176 +0000 UTC m=+2843.502230437" watchObservedRunningTime="2026-07-22 05:20:28.45714939 +0000 UTC m=+2843.505417858" Jul 22 05:20:30.015659 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:30.015623 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pxjdd_618eb713-6cb2-46cc-89aa-2e84b3eb848b/dns/0.log" Jul 22 05:20:30.035762 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:30.035722 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pxjdd_618eb713-6cb2-46cc-89aa-2e84b3eb848b/kube-rbac-proxy/0.log" Jul 22 05:20:30.058788 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:30.058758 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7266c_4cd9108c-5501-48c3-a5ea-879f5e0a727f/dns-node-resolver/0.log" Jul 22 05:20:30.660179 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:30.660146 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6dd6957979-g82wp_3127d883-9f15-4947-857d-47e76ff4d463/registry/0.log" Jul 22 05:20:30.730993 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:30.730952 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wfvlw_64d5801c-fe21-44dd-af4e-de832e04007c/node-ca/0.log" Jul 22 05:20:31.697106 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:31.697069 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678j5fmj_03259dd4-d21f-454a-85c5-b2ea70adfa79/istio-proxy/0.log" Jul 22 05:20:31.868114 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:31.868080 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d644bdf8d-qgsln_a7476468-64fe-4598-ba2b-e5775a458b0d/kube-auth-proxy/0.log" Jul 22 05:20:31.895705 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:31.895658 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/0.log" Jul 22 05:20:31.954043 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:31.953933 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/1.log" Jul 22 05:20:32.775421 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:32.775383 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b6c5q_13b2bd6f-8ed7-4e1f-9920-5dba9a915009/serve-healthcheck-canary/0.log" Jul 22 05:20:33.379793 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:33.379749 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmm6_80f60688-875a-4d8a-8cf7-9f7f1eabaf7a/kube-rbac-proxy/0.log" Jul 22 05:20:33.402444 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:33.402405 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmm6_80f60688-875a-4d8a-8cf7-9f7f1eabaf7a/exporter/0.log" Jul 22 05:20:33.424932 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:33.424894 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mvmm6_80f60688-875a-4d8a-8cf7-9f7f1eabaf7a/extractor/0.log" Jul 22 05:20:34.459425 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:34.459390 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-84vmg/perf-node-gather-daemonset-xxdnd" Jul 22 05:20:35.652466 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:35.652427 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wxltv_241e0eae-d237-46f1-9151-c9302d843c75/manager/0.log" Jul 22 05:20:36.008049 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.007546 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4bfl5_b81bea95-d97a-4049-9f97-c79f21dbf0f3/manager/0.log" Jul 22 05:20:36.243057 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.242825 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-865d8b7674-wczgw_00019a52-7e92-4e78-9628-788953f92677/manager/0.log" Jul 22 05:20:36.265212 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.265105 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r7cm5_2bc3a940-3aa9-429c-b1c1-ad00677d8f53/server/0.log" Jul 22 05:20:36.385803 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.385764 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-dqv4n_b89822bc-24c2-4c21-b858-0d592985075f/manager/0.log" Jul 22 05:20:36.564079 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.563963 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5567574457-s8tcd_31944654-99d5-4b4b-9b2a-fe3b7db42e29/manager/0.log" Jul 22 05:20:36.583940 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:36.583903 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5567574457-s8tcd_31944654-99d5-4b4b-9b2a-fe3b7db42e29/copy-manifests/0.log" Jul 22 05:20:38.146076 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:38.146012 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9dcf4_f10ad3f0-2113-457a-ba79-ee7d3f30b45e/openshift-lws-operator/0.log" Jul 22 05:20:38.722477 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:38.722433 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678j5fmj_03259dd4-d21f-454a-85c5-b2ea70adfa79/istio-proxy/0.log" Jul 22 05:20:38.881074 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:38.881016 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d644bdf8d-qgsln_a7476468-64fe-4598-ba2b-e5775a458b0d/kube-auth-proxy/0.log" Jul 22 05:20:38.900337 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:38.900302 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/0.log" Jul 22 05:20:38.959660 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:38.959619 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/1.log" Jul 22 05:20:40.262337 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:40.262295 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678j5fmj_03259dd4-d21f-454a-85c5-b2ea70adfa79/istio-proxy/0.log" Jul 22 05:20:40.422793 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:40.422755 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-d644bdf8d-qgsln_a7476468-64fe-4598-ba2b-e5775a458b0d/kube-auth-proxy/0.log" Jul 22 05:20:40.446138 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:40.446095 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/0.log" Jul 22 05:20:40.503235 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:40.503201 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jspkp_00d98c58-f3ca-4f07-a2bd-c77963aefb13/istio-proxy/1.log" Jul 22 05:20:44.637589 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:44.637552 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-69mng_5dd21a8c-358a-4a4a-adff-48c634977ea3/kube-multus/0.log" Jul 22 05:20:44.963454 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:44.963371 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/kube-multus-additional-cni-plugins/0.log" Jul 22 05:20:44.983662 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:44.983630 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/egress-router-binary-copy/0.log" Jul 22 05:20:45.005718 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.005695 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/cni-plugins/0.log" Jul 22 05:20:45.025901 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.025876 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/bond-cni-plugin/0.log" Jul 22 05:20:45.045848 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.045817 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/routeoverride-cni/0.log" Jul 22 05:20:45.066114 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.066082 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/whereabouts-cni-bincopy/0.log" Jul 22 05:20:45.086274 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.086242 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nzzlb_b05cd5e1-cdc3-43bb-90dd-9ae6d7961ad4/whereabouts-cni/0.log" Jul 22 05:20:45.172354 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.172322 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bwfx_379b2ddd-c7ae-4cda-ab35-996636b9eb3c/network-metrics-daemon/0.log" Jul 22 05:20:45.187365 ip-10-0-133-75 kubenswrapper[2555]: I0722 05:20:45.187333 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bwfx_379b2ddd-c7ae-4cda-ab35-996636b9eb3c/kube-rbac-proxy/0.log"