Jul 14 17:02:53.064452 ip-10-0-129-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 14 17:02:53.064461 ip-10-0-129-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 14 17:02:53.064469 ip-10-0-129-213 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 14 17:02:53.064695 ip-10-0-129-213 systemd[1]: Failed to start Kubernetes Kubelet. Jul 14 17:03:03.195292 ip-10-0-129-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 14 17:03:03.195310 ip-10-0-129-213 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 75dbbc53b2304c7ebf56702d6a8dc5ee -- Jul 14 17:05:06.318922 ip-10-0-129-213 systemd[1]: Starting Kubernetes Kubelet... Jul 14 17:05:06.813362 ip-10-0-129-213 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:05:06.813362 ip-10-0-129-213 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 17:05:06.813362 ip-10-0-129-213 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:05:06.813362 ip-10-0-129-213 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 17:05:06.813362 ip-10-0-129-213 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:05:06.817841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.817549 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825491 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825510 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825514 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825517 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825521 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825524 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825528 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825531 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825533 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825536 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825539 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825542 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825544 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825547 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825549 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825552 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825554 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825557 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825565 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:06.923958 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825567 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825570 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825588 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825591 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825594 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825596 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825599 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825602 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825605 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825607 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825611 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825614 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825617 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825619 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825622 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825624 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825628 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825630 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825633 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825635 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:06.935627 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825638 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825641 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825643 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825646 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825648 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825651 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825653 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825656 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825658 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825661 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825663 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825666 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825670 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825673 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825675 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825679 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825683 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825687 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825690 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:06.936395 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825693 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825697 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825700 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825703 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825706 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825709 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825712 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825714 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825717 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825720 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825723 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825725 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825728 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825730 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825733 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825735 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825738 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825740 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825743 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825746 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:06.938315 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825748 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825751 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825754 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825758 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825761 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825764 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825767 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825770 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825772 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825776 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825779 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825781 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825784 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825787 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825789 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825792 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825794 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825798 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825802 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:06.939073 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825805 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.825808 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826543 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826550 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826555 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826558 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826561 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826564 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826567 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826569 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826586 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826589 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826592 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826596 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826600 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826603 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826606 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826609 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826612 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:06.941016 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826616 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826618 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826622 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826625 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826627 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826630 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826633 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826635 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826638 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826640 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826643 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826647 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826649 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826652 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826654 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826657 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826660 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826663 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826666 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826668 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:06.941816 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826671 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826673 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826675 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826678 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826680 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826683 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826685 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826689 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826692 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826695 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826697 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826700 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826702 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826706 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826708 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826711 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826713 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826716 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826718 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826721 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:06.944063 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826723 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826726 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826728 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826731 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826733 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826736 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826740 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826744 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826747 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826750 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826754 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826757 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826760 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826763 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826766 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826768 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826771 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826773 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826776 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826779 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:06.944827 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826782 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826786 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826789 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826792 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826794 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826797 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826801 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826803 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826806 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826809 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826811 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826814 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826816 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826819 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826821 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826824 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826827 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826830 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826833 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826835 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:06.947904 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826838 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.826841 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827269 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827275 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827279 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827283 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827286 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827290 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827293 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827296 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827299 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827302 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827305 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827307 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827311 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827314 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827317 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827320 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827329 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:06.948678 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827333 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827337 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827339 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827342 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827345 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827347 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827350 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827352 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827355 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827357 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827360 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827363 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827366 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827368 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827370 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827373 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827376 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827379 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827381 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:06.951387 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827384 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827387 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827391 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827394 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827397 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827400 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827403 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827406 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827408 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827411 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827414 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827417 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827419 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827422 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827425 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827428 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827430 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827433 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827436 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:06.952159 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827438 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827444 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827446 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827449 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827451 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827455 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827458 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827461 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827464 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827466 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827469 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827473 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827476 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827478 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827481 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827483 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827486 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827488 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827491 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827493 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827495 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:06.955969 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827498 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827501 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827503 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827506 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827508 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827511 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827514 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827516 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827519 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827523 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827526 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827529 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827533 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827536 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827539 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827541 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827545 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827548 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827550 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827553 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:06.956749 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827555 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827558 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.827561 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827654 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827666 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827675 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827682 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827688 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827693 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827699 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827710 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827715 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827721 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827724 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827728 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827731 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827734 2574 flags.go:64] FLAG: --cgroup-root="" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827737 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827740 2574 flags.go:64] FLAG: --client-ca-file="" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827744 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827747 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827752 2574 flags.go:64] FLAG: --cluster-domain="" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827755 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 17:05:07.053892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827758 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827763 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827767 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827771 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827774 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827780 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827783 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827786 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827790 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827793 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827796 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827799 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827803 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827807 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827810 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827813 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827816 2574 flags.go:64] FLAG: --enable-server="true" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827818 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827824 2574 flags.go:64] FLAG: --event-burst="100" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827827 2574 flags.go:64] FLAG: --event-qps="50" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827830 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827833 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827836 2574 flags.go:64] FLAG: --eviction-hard="" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827839 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 17:05:07.054662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827842 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827845 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827848 2574 flags.go:64] FLAG: --eviction-soft="" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827851 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827854 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827858 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827861 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827864 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827867 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827871 2574 flags.go:64] FLAG: --feature-gates="" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827875 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827878 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827881 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827886 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827889 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827892 2574 flags.go:64] FLAG: --help="false" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827895 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827898 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827902 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827905 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827908 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827912 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827915 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 17:05:07.057799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827918 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827921 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827923 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827926 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827929 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827932 2574 flags.go:64] FLAG: --kube-reserved="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827935 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827938 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827941 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827943 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827946 2574 flags.go:64] FLAG: --lock-file="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827949 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827952 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827955 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827961 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827964 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827967 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827970 2574 flags.go:64] FLAG: --logging-format="text" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827977 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827980 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827983 2574 flags.go:64] FLAG: --manifest-url="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827986 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827991 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827994 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.827999 2574 flags.go:64] FLAG: --max-pods="110" Jul 14 17:05:07.058556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828001 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828004 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828007 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828010 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828013 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828016 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828019 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828032 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828035 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828038 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828041 2574 flags.go:64] FLAG: --pod-cidr="" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828043 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828049 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828051 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828054 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828057 2574 flags.go:64] FLAG: --port="10250" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828060 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828063 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c330eb522018a984" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828066 2574 flags.go:64] FLAG: --qos-reserved="" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828069 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828072 2574 flags.go:64] FLAG: --register-node="true" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828075 2574 flags.go:64] FLAG: --register-with-taints="" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828078 2574 flags.go:64] FLAG: --registry-burst="10" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828082 2574 flags.go:64] FLAG: --registry-qps="5" Jul 14 17:05:07.061284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828085 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828089 2574 flags.go:64] FLAG: --reserved-memory="" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828093 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828096 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828099 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828102 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828106 2574 flags.go:64] FLAG: --runonce="false" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828109 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828112 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828115 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828118 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828121 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828124 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828128 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828131 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828134 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828141 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828144 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828147 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828150 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828153 2574 flags.go:64] FLAG: --system-cgroups="" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828156 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828161 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828164 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828167 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 17:05:07.062066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828171 2574 flags.go:64] FLAG: --tls-min-version="" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828174 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828177 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828181 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828183 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828186 2574 flags.go:64] FLAG: --v="2" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828190 2574 flags.go:64] FLAG: --version="false" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828195 2574 flags.go:64] FLAG: --vmodule="" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828200 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.828203 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828339 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828344 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828347 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828352 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828356 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828358 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828361 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828364 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828366 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828369 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828372 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828374 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:07.065993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828377 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828380 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828382 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828386 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828390 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828393 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828395 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828398 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828401 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828403 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828405 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828408 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828411 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828414 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828416 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828419 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828421 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828424 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828428 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828431 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:07.066757 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828433 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828435 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828438 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828442 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828445 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828448 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828451 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828453 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828456 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828458 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828461 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828465 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828468 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828471 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828474 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828477 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828480 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828482 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828485 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:07.070652 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828488 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828491 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828493 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828496 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828498 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828501 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828504 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828506 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828509 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828511 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828514 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828518 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828522 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828525 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828528 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828531 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828535 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828538 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:07.071410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828541 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828543 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828550 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828553 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828556 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828559 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828561 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828564 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828567 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828569 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828585 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828588 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828591 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828593 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828596 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828599 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828601 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828604 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828607 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828609 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:07.074479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828612 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828614 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828617 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828620 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828622 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828626 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828629 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828632 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828635 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.828638 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.829451 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.839203 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.839224 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.840079 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:06.840087 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 17:05:07.075248 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840178 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840184 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840187 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840191 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840194 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840197 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840200 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840202 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840205 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840208 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840210 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840213 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840216 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840219 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840222 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840225 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840227 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840230 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840233 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840235 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:07.077978 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840238 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:07.078781 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840242 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:07.078781 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840247 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:07.078781 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840250 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:07.434045 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:06.840253 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434079 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434102 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434107 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434112 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434116 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434122 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:07.434123 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434127 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434132 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434136 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434140 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434144 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434148 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434152 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434156 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434160 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434164 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434168 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434172 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434175 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434180 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434184 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434188 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434193 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434197 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434200 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434204 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:07.434341 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434208 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434213 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434217 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434221 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434229 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434235 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434258 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434264 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434268 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434272 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434276 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434280 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434283 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434287 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434292 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434296 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434300 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434304 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434308 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:07.434902 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434312 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434316 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434321 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434325 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434329 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434333 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434337 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434340 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434345 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434349 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434353 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434357 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434361 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434364 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434369 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434373 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434377 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434381 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434384 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434388 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:07.435436 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434399 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434404 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434408 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434412 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434415 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434420 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434427 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434434 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434438 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.434448 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434738 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434747 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434752 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434757 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434761 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:05:07.435988 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434764 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434769 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434774 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434778 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434782 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434787 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434791 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434794 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434798 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434802 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434805 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434809 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434814 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434818 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434822 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434826 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434830 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434836 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434849 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:05:07.436398 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434854 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434859 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434863 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434867 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434872 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434877 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434882 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434887 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434891 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434895 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434899 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434903 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434906 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434910 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434914 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434918 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434923 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434928 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434934 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:05:07.436926 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434938 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434942 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434946 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434949 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434953 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434957 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434961 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434965 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434969 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434973 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434977 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434981 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434985 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.434995 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435001 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435004 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435008 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435012 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435017 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435021 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:05:07.437475 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435025 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435036 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435041 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435045 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435049 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435053 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435056 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435060 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435064 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435068 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435072 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435076 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435079 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435083 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435086 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435090 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435094 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435098 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435102 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435105 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:05:07.438028 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435109 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435113 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435116 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435120 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435124 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435128 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435142 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435146 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435150 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435154 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435160 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435165 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435170 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435174 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435179 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:05:07.438516 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:07.435184 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:05:07.438926 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.435191 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:05:07.438926 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.435378 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 17:05:07.447434 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.447416 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 17:05:07.449068 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.449055 2574 server.go:1019] "Starting client certificate rotation" Jul 14 17:05:07.449189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.449167 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:05:07.449219 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.449211 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:05:07.451793 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.451779 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:05:07.457947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.457912 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:05:07.472318 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.472289 2574 log.go:25] "Validated CRI v1 runtime API" Jul 14 17:05:07.474444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.474429 2574 log.go:25] "Validated CRI v1 image API" Jul 14 17:05:07.475357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.475341 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 17:05:07.479926 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.479892 2574 fs.go:135] Filesystem UUIDs: map[47ec7742-e263-4bd4-850d-2f242b8eba53:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 cfcc572b-3158-425f-9475-b36bf67ca2c6:/dev/nvme0n1p4] Jul 14 17:05:07.480008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.479918 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 17:05:07.484790 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.484771 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:05:07.485694 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.485564 2574 manager.go:216] Machine: {Timestamp:2026-07-14 17:05:07.484167054 +0000 UTC m=+1.007107818 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097532 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26ffcde8dd737fb8a1b84a49503ad9 SystemUUID:ec26ffcd-e8dd-737f-b8a1-b84a49503ad9 BootID:75dbbc53-b230-4c7e-bf56-702d6a8dc5ee Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0c:73:bb:5a:1b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0c:73:bb:5a:1b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:43:4d:55:1a:2a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 17:05:07.485694 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.485693 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 17:05:07.485842 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.485815 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 17:05:07.488553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.488522 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 17:05:07.488722 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.488558 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 17:05:07.489753 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.489742 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 17:05:07.489794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.489754 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 17:05:07.489794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.489762 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:05:07.490419 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.490408 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:05:07.491665 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.491655 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 17:05:07.494421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494409 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:05:07.494586 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494563 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 17:05:07.494625 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494605 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 14 17:05:07.494625 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494616 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 17:05:07.494676 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494627 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 17:05:07.494676 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494635 2574 kubelet.go:398] "Adding apiserver pod source" Jul 14 17:05:07.494676 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.494645 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 17:05:07.495064 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.495051 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:05:07.495121 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.495072 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:05:07.497412 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.497392 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 17:05:07.500768 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.500742 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 17:05:07.500852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.500784 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 17:05:07.501015 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501001 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 17:05:07.501091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501025 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 17:05:07.501091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501052 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 17:05:07.501091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501063 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 17:05:07.501091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501076 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 17:05:07.501091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501086 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 17:05:07.501527 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501516 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 17:05:07.501593 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501528 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 17:05:07.501593 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501557 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 17:05:07.501684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501603 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 17:05:07.501684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501617 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 17:05:07.501684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.501628 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 17:05:07.502945 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.502931 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 17:05:07.502994 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.502950 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 17:05:07.504118 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.504093 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 17:05:07.504215 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.504093 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 17:05:07.506436 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.506423 2574 server.go:1268] "Started kubelet" Jul 14 17:05:07.506550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.506522 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 17:05:07.506618 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.506507 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 17:05:07.506665 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.506633 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 17:05:07.507567 ip-10-0-129-213 systemd[1]: Started Kubernetes Kubelet. Jul 14 17:05:07.507936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.507641 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 17:05:07.508636 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.508623 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 14 17:05:07.511614 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.511591 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wnrl2" Jul 14 17:05:07.512859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.512840 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 17:05:07.512956 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.512851 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 17:05:07.513159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513138 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 17:05:07.513159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513152 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 17:05:07.513297 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.513258 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:07.513796 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513779 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 17:05:07.513869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513799 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 17:05:07.513869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513779 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 17:05:07.513934 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513893 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 14 17:05:07.513934 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.513901 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 14 17:05:07.513934 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.512941 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-213.ec2.internal.18c236d8e0e9ca54 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-213.ec2.internal,UID:ip-10-0-129-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-213.ec2.internal,},FirstTimestamp:2026-07-14 17:05:07.506391636 +0000 UTC m=+1.029332400,LastTimestamp:2026-07-14 17:05:07.506391636 +0000 UTC m=+1.029332400,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-213.ec2.internal,}" Jul 14 17:05:07.514032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514007 2574 factory.go:55] Registering systemd factory Jul 14 17:05:07.514032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514024 2574 factory.go:223] Registration of the systemd container factory successfully Jul 14 17:05:07.514253 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514239 2574 factory.go:153] Registering CRI-O factory Jul 14 17:05:07.514253 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514251 2574 factory.go:223] Registration of the crio container factory successfully Jul 14 17:05:07.514386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514307 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 17:05:07.514386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514326 2574 factory.go:103] Registering Raw factory Jul 14 17:05:07.514386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.514341 2574 manager.go:1196] Started watching for new ooms in manager Jul 14 17:05:07.515491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.515476 2574 manager.go:349] Starting recovery of all containers Jul 14 17:05:07.518191 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.518145 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 17:05:07.518270 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.518211 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 17:05:07.520798 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.520773 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wnrl2" Jul 14 17:05:07.522480 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.522445 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 17:05:07.527650 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.527344 2574 manager.go:354] Recovery completed Jul 14 17:05:07.530148 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.530123 2574 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 14 17:05:07.533089 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.533076 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.534936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.534914 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.535009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.534959 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.535009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.534970 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.535508 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.535489 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 17:05:07.535508 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.535505 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.535631 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.535506 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 17:05:07.535631 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.535541 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:05:07.536247 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536230 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.536299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536270 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.536299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536280 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.536767 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536754 2574 policy_none.go:49] "None policy: Start" Jul 14 17:05:07.536767 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536770 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 17:05:07.536869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.536780 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 17:05:07.536961 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.536872 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-213.ec2.internal.18c236d8e29d555c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-213.ec2.internal,UID:ip-10-0-129-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-213.ec2.internal,},FirstTimestamp:2026-07-14 17:05:07.534935388 +0000 UTC m=+1.057876152,LastTimestamp:2026-07-14 17:05:07.534935388 +0000 UTC m=+1.057876152,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-213.ec2.internal,}" Jul 14 17:05:07.537739 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.537728 2574 policy_none.go:47] "Start" Jul 14 17:05:07.575219 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575197 2574 manager.go:337] "Starting Device Plugin manager" Jul 14 17:05:07.575377 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.575236 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 17:05:07.575377 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575246 2574 server.go:87] "Starting device plugin registration server" Jul 14 17:05:07.575543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575531 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 17:05:07.575595 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575546 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 17:05:07.575701 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575680 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 17:05:07.575813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575789 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 17:05:07.575813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.575799 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 17:05:07.576336 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.576315 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 17:05:07.576425 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.576365 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:07.676170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.676083 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.677261 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.677144 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.677261 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.677175 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.677261 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.677186 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.677261 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.677213 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.685912 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.685894 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.685997 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.685918 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-213.ec2.internal\": node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:07.686049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.686033 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 17:05:07.686082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.686071 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 17:05:07.686112 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.686094 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 17:05:07.686157 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.686127 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 17:05:07.688326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.688308 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:07.733174 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.733150 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:07.787144 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.787116 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal"] Jul 14 17:05:07.787241 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.787223 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.788189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.788173 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.788283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.788202 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.788283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.788212 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.789544 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.789532 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.789653 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.789638 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.789689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.789673 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.790260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790241 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.790345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790268 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.790345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790278 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.790345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790242 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.790345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790338 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.790473 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.790349 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.791432 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.791417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.791481 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.791444 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:05:07.792033 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.792016 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:05:07.792121 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.792042 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:05:07.792121 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.792072 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:05:07.815842 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.815809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d98126d381549eedd58b496d19efe54c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-213.ec2.internal\" (UID: \"d98126d381549eedd58b496d19efe54c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.815842 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.815843 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.816371 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.815863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.820404 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.820387 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-213.ec2.internal\" not found" node="ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.824715 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.824697 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-213.ec2.internal\" not found" node="ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.833275 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.833252 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:07.916325 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d98126d381549eedd58b496d19efe54c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-213.ec2.internal\" (UID: \"d98126d381549eedd58b496d19efe54c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.916474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916333 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.916474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916364 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.916474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.916474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916288 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d98126d381549eedd58b496d19efe54c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-213.ec2.internal\" (UID: \"d98126d381549eedd58b496d19efe54c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.916474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:07.916445 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1438194959089c40336a26a00fa74afd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal\" (UID: \"1438194959089c40336a26a00fa74afd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:07.933422 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:07.933329 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.034167 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.034134 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.122668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.122637 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:08.127320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.127301 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:08.134880 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.134861 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.235511 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.235407 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.336040 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.335994 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.436536 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.436496 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.448656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.448633 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 17:05:08.448794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.448778 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:05:08.524559 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.524458 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:00:07 +0000 UTC" deadline="2028-01-04 14:40:28.590982171 +0000 UTC" Jul 14 17:05:08.524559 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.524494 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12933h35m20.06649116s" Jul 14 17:05:08.536647 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.536616 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.637299 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.637253 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.737794 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.737749 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.767252 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.767227 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:08.825537 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:08.825482 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:08.838248 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.838212 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:08.938740 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:08.938703 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-213.ec2.internal\" not found" Jul 14 17:05:09.017091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.017061 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:09.114273 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.114247 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" Jul 14 17:05:09.117406 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.117379 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1438194959089c40336a26a00fa74afd.slice/crio-95b17b289145961a044ab7ab0557a784d34689819c5c1678b0abd6efe39e3b40 WatchSource:0}: Error finding container 95b17b289145961a044ab7ab0557a784d34689819c5c1678b0abd6efe39e3b40: Status 404 returned error can't find the container with id 95b17b289145961a044ab7ab0557a784d34689819c5c1678b0abd6efe39e3b40 Jul 14 17:05:09.121670 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.121655 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:05:09.124161 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.124141 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:05:09.125191 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.125176 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" Jul 14 17:05:09.135970 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.135950 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:05:09.495326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.495224 2574 apiserver.go:52] "Watching apiserver" Jul 14 17:05:09.504560 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.504540 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 17:05:09.504899 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.504877 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-46tkm","kube-system/konnectivity-agent-klggv","openshift-multus/network-metrics-daemon-bmf4s","openshift-network-diagnostics/network-check-target-nd78m","openshift-network-operator/iptables-alerter-5xq9d","openshift-ovn-kubernetes/ovnkube-node-tj95b","kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x","openshift-cluster-node-tuning-operator/tuned-fnv55","openshift-dns/node-resolver-csj2m","openshift-image-registry/node-ca-4sw8m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal","openshift-multus/multus-68j4h","openshift-multus/multus-additional-cni-plugins-48gdt"] Jul 14 17:05:09.507157 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.507136 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.508179 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.508162 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.509295 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.509273 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.509448 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.509413 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:09.509634 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.509620 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xs6gm\"" Jul 14 17:05:09.509704 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.509638 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.510224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510210 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.510424 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510410 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:09.510500 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.510479 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:09.510545 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510501 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.510705 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510693 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.510878 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510861 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 17:05:09.510935 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.510922 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 17:05:09.511031 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.511014 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5b8xk\"" Jul 14 17:05:09.511460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.511445 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.512915 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.512897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.513486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.513465 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 17:05:09.514067 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.514048 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 17:05:09.514148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.514139 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.514205 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.514177 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2zr5h\"" Jul 14 17:05:09.514332 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.514289 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.514551 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.514534 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.515083 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.515066 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 17:05:09.515083 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.515072 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.515466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.515453 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.515802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.515788 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.516683 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.516661 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 17:05:09.516769 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.516679 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xjg44\"" Jul 14 17:05:09.516769 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.516688 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 17:05:09.516769 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.516727 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 17:05:09.517114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517093 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.517191 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517124 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.517243 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517192 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gdjp2\"" Jul 14 17:05:09.517243 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517210 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.517901 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517884 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 17:05:09.517990 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.517928 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 17:05:09.518246 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.518225 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cdmfg\"" Jul 14 17:05:09.518321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.518298 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.518374 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.518353 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:09.519409 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.519393 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.519858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.519841 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 17:05:09.519946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.519871 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bvg2x\"" Jul 14 17:05:09.520029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.519999 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 17:05:09.520640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.520622 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.521599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.521564 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 17:05:09.523663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.521859 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.523663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.522081 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.523663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.522099 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lsrh4\"" Jul 14 17:05:09.523663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.523234 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 17:05:09.523663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.523410 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 17:05:09.523867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.523759 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 17:05:09.523867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.523763 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jvm84\"" Jul 14 17:05:09.524978 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.524960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-lib-modules\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.524995 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87rlk\" (UniqueName: \"kubernetes.io/projected/1df5604a-b419-4223-b152-acfd179dfb22-kube-api-access-87rlk\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-os-release\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-var-lib-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-cni-binary-copy\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525158 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525122 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-iptables-alerter-script\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.525158 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525149 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-k8s-cni-cncf-io\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525173 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-kubelet\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525199 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:09.525294 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88367bbb-0442-4660-9d3f-b7cc31e51be1-tmp-dir\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.525294 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525294 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-hostroot\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-ovn\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525340 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-node-log\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525384 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88367bbb-0442-4660-9d3f-b7cc31e51be1-hosts-file\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.525526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525407 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.525526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525430 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-multus\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525453 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-systemd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525486 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-config\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-env-overrides\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525566 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-host-slash\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525606 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysconfig\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525637 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-systemd\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-run\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-multus-daemon-config\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525688 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-var-lib-kubelet\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.525714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525703 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-netd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-script-lib\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525768 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-sys\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525802 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-socket-dir-parent\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525830 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg67t\" (UniqueName: \"kubernetes.io/projected/e1090c57-e98d-4da2-bcc1-23cae48d366e-kube-api-access-zg67t\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525879 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plvmt\" (UniqueName: \"kubernetes.io/projected/88367bbb-0442-4660-9d3f-b7cc31e51be1-kube-api-access-plvmt\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7zbt\" (UniqueName: \"kubernetes.io/projected/7ceb8907-d175-4f86-957c-e924c6093676-kube-api-access-s7zbt\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525953 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-modprobe-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525967 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-cnibin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.525987 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-netns\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-etc-tuned\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-system-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526038 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-bin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-multus-certs\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526067 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-netns\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526089 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-tmp\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526120 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-etc-kubernetes\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526134 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovn-node-metrics-cert\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-kubernetes\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-host\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526188 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-conf-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swhps\" (UniqueName: \"kubernetes.io/projected/bd92ffff-5264-4b21-832d-88a3a9581098-kube-api-access-swhps\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526223 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whcvt\" (UniqueName: \"kubernetes.io/projected/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-kube-api-access-whcvt\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-slash\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526273 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-systemd-units\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526345 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-etc-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526369 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-log-socket\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526399 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-conf\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.526520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526427 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-kubelet\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.527047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.526450 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-bin\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.528729 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.528714 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:05:09.548732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.548709 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4t4tf" Jul 14 17:05:09.557902 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.557876 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4t4tf" Jul 14 17:05:09.615457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.615418 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 17:05:09.626880 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-netd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.626880 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626881 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-script-lib\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-os-release\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626931 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-binary-copy\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/425266e1-079d-46b8-8366-ce4b5651f269-host\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-netd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.626968 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-socket-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-sys\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627044 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-socket-dir-parent\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-sys\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.627122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627123 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627124 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-socket-dir-parent\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg67t\" (UniqueName: \"kubernetes.io/projected/e1090c57-e98d-4da2-bcc1-23cae48d366e-kube-api-access-zg67t\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plvmt\" (UniqueName: \"kubernetes.io/projected/88367bbb-0442-4660-9d3f-b7cc31e51be1-kube-api-access-plvmt\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627206 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7zbt\" (UniqueName: \"kubernetes.io/projected/7ceb8907-d175-4f86-957c-e924c6093676-kube-api-access-s7zbt\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-modprobe-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-ovn-kubernetes\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627260 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-cnibin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-netns\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627373 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-netns\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-kubelet-config\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627344 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-cnibin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627406 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-modprobe-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627415 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-dbus\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627451 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-etc-tuned\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-system-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627526 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-bin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.627542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-multus-certs\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-script-lib\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-multus-certs\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627603 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-bin\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627554 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-system-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627642 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-netns\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627690 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-run-netns\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/425266e1-079d-46b8-8366-ce4b5651f269-serviceca\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-sys-fs\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-tmp\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627787 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-etc-kubernetes\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovn-node-metrics-cert\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627819 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70960940-3298-4dee-842a-61b2d4494306-agent-certs\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-device-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627904 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-kubernetes\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.628290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627883 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-etc-kubernetes\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627933 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-host\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627958 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-kubernetes\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627972 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-conf-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.627982 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-host\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swhps\" (UniqueName: \"kubernetes.io/projected/bd92ffff-5264-4b21-832d-88a3a9581098-kube-api-access-swhps\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628033 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whcvt\" (UniqueName: \"kubernetes.io/projected/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-kube-api-access-whcvt\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628058 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-slash\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-conf-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfrb4\" (UniqueName: \"kubernetes.io/projected/425266e1-079d-46b8-8366-ce4b5651f269-kube-api-access-hfrb4\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-slash\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628189 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-systemd-units\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-etc-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-log-socket\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-systemd-units\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq729\" (UniqueName: \"kubernetes.io/projected/68c24a22-c9e1-4d60-bae0-819f1225272a-kube-api-access-cq729\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.629116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628318 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-etc-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-conf\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-log-socket\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628300 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-d\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628349 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-kubelet\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628377 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-bin\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-kubelet\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628400 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628413 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-host-cni-bin\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628426 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-lib-modules\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628433 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysctl-conf\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628441 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87rlk\" (UniqueName: \"kubernetes.io/projected/1df5604a-b419-4223-b152-acfd179dfb22-kube-api-access-87rlk\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628472 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-os-release\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-var-lib-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628519 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-lib-modules\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628526 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-tuning-conf-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-cni-binary-copy\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.629914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628555 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-var-lib-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628589 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-cnibin\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-os-release\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xp4v\" (UniqueName: \"kubernetes.io/projected/3e6f316d-5b8a-4824-9f6e-dd7373442614-kube-api-access-9xp4v\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628667 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-iptables-alerter-script\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-k8s-cni-cncf-io\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628718 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-kubelet\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628743 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628766 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88367bbb-0442-4660-9d3f-b7cc31e51be1-tmp-dir\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628766 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-run-k8s-cni-cncf-io\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628787 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628812 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-hostroot\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-kubelet\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629020 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-multus-cni-dir\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629068 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-hostroot\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629125 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88367bbb-0442-4660-9d3f-b7cc31e51be1-tmp-dir\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629158 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-cni-binary-copy\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629182 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-ovn\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.630762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.628834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-ovn\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629224 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-node-log\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629237 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-iptables-alerter-script\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629289 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88367bbb-0442-4660-9d3f-b7cc31e51be1-hosts-file\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629292 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-node-log\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629328 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629361 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-multus\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-systemd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.629392 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-config\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629418 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bd92ffff-5264-4b21-832d-88a3a9581098-host-var-lib-cni-multus\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629441 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-env-overrides\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629475 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-system-cni-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629478 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-systemd\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629387 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/88367bbb-0442-4660-9d3f-b7cc31e51be1-hosts-file\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.629502 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:10.129468477 +0000 UTC m=+3.652409231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:09.631352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629557 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629597 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70960940-3298-4dee-842a-61b2d4494306-konnectivity-ca\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629624 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-registration-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629676 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-host-slash\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629715 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysconfig\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629739 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-systemd\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629741 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1090c57-e98d-4da2-bcc1-23cae48d366e-run-openvswitch\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-run\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629790 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-host-slash\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-multus-daemon-config\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629879 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-var-lib-kubelet\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629912 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-systemd\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629933 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-env-overrides\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629997 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-run\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629997 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-var-lib-kubelet\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.629998 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1df5604a-b419-4223-b152-acfd179dfb22-etc-sysconfig\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.631838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.630077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovnkube-config\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.632299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.630367 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bd92ffff-5264-4b21-832d-88a3a9581098-multus-daemon-config\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.632299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.631080 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-tmp\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.632299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.631110 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1df5604a-b419-4223-b152-acfd179dfb22-etc-tuned\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.632299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.631288 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1090c57-e98d-4da2-bcc1-23cae48d366e-ovn-node-metrics-cert\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.637700 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.637675 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:09.637761 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.637704 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:09.637761 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.637714 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:09.637832 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.637786 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:10.137769339 +0000 UTC m=+3.660710112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:09.640119 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.640093 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87rlk\" (UniqueName: \"kubernetes.io/projected/1df5604a-b419-4223-b152-acfd179dfb22-kube-api-access-87rlk\") pod \"tuned-fnv55\" (UID: \"1df5604a-b419-4223-b152-acfd179dfb22\") " pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.640241 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.640181 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg67t\" (UniqueName: \"kubernetes.io/projected/e1090c57-e98d-4da2-bcc1-23cae48d366e-kube-api-access-zg67t\") pod \"ovnkube-node-tj95b\" (UID: \"e1090c57-e98d-4da2-bcc1-23cae48d366e\") " pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.640342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.640322 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swhps\" (UniqueName: \"kubernetes.io/projected/bd92ffff-5264-4b21-832d-88a3a9581098-kube-api-access-swhps\") pod \"multus-68j4h\" (UID: \"bd92ffff-5264-4b21-832d-88a3a9581098\") " pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.641158 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.641136 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whcvt\" (UniqueName: \"kubernetes.io/projected/11f3d5e8-a605-4a7d-8b81-5752720e1eb8-kube-api-access-whcvt\") pod \"iptables-alerter-5xq9d\" (UID: \"11f3d5e8-a605-4a7d-8b81-5752720e1eb8\") " pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.641496 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.641482 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plvmt\" (UniqueName: \"kubernetes.io/projected/88367bbb-0442-4660-9d3f-b7cc31e51be1-kube-api-access-plvmt\") pod \"node-resolver-csj2m\" (UID: \"88367bbb-0442-4660-9d3f-b7cc31e51be1\") " pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.641566 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.641542 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7zbt\" (UniqueName: \"kubernetes.io/projected/7ceb8907-d175-4f86-957c-e924c6093676-kube-api-access-s7zbt\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:09.675779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.675745 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:05:09.690186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.690138 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" event={"ID":"1438194959089c40336a26a00fa74afd","Type":"ContainerStarted","Data":"95b17b289145961a044ab7ab0557a784d34689819c5c1678b0abd6efe39e3b40"} Jul 14 17:05:09.730609 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730553 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-os-release\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.730609 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-binary-copy\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.730810 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-os-release\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.730810 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730757 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/425266e1-079d-46b8-8366-ce4b5651f269-host\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.730810 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730789 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-socket-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.730953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-kubelet-config\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.730953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730847 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-dbus\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.730953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730889 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/425266e1-079d-46b8-8366-ce4b5651f269-host\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.730953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.730953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730953 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/425266e1-079d-46b8-8366-ce4b5651f269-serviceca\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730955 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-socket-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730946 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-kubelet-config\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.730976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-sys-fs\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731036 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-sys-fs\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70960940-3298-4dee-842a-61b2d4494306-agent-certs\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.731039 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:09.731168 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:10.231149689 +0000 UTC m=+3.754090446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:09.731186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dba23291-95f1-4b81-9c35-80a0955c5935-dbus\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-device-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731268 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-device-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731348 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfrb4\" (UniqueName: \"kubernetes.io/projected/425266e1-079d-46b8-8366-ce4b5651f269-kube-api-access-hfrb4\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731391 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq729\" (UniqueName: \"kubernetes.io/projected/68c24a22-c9e1-4d60-bae0-819f1225272a-kube-api-access-cq729\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731422 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731454 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-tuning-conf-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731478 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-cnibin\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp4v\" (UniqueName: \"kubernetes.io/projected/3e6f316d-5b8a-4824-9f6e-dd7373442614-kube-api-access-9xp4v\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731606 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/425266e1-079d-46b8-8366-ce4b5651f269-serviceca\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.731649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731638 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-system-cni-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731666 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70960940-3298-4dee-842a-61b2d4494306-konnectivity-ca\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731683 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-cnibin\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-tuning-conf-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731702 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-registration-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731738 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e6f316d-5b8a-4824-9f6e-dd7373442614-system-cni-dir\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731804 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-registration-dir\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731897 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/68c24a22-c9e1-4d60-bae0-819f1225272a-etc-selinux\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.731920 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.732131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.732235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.732213 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70960940-3298-4dee-842a-61b2d4494306-konnectivity-ca\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.732689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.732326 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e6f316d-5b8a-4824-9f6e-dd7373442614-cni-binary-copy\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.734297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.734275 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70960940-3298-4dee-842a-61b2d4494306-agent-certs\") pod \"konnectivity-agent-klggv\" (UID: \"70960940-3298-4dee-842a-61b2d4494306\") " pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.740799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.740767 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xp4v\" (UniqueName: \"kubernetes.io/projected/3e6f316d-5b8a-4824-9f6e-dd7373442614-kube-api-access-9xp4v\") pod \"multus-additional-cni-plugins-48gdt\" (UID: \"3e6f316d-5b8a-4824-9f6e-dd7373442614\") " pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.740996 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.740962 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfrb4\" (UniqueName: \"kubernetes.io/projected/425266e1-079d-46b8-8366-ce4b5651f269-kube-api-access-hfrb4\") pod \"node-ca-4sw8m\" (UID: \"425266e1-079d-46b8-8366-ce4b5651f269\") " pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.741087 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.741046 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq729\" (UniqueName: \"kubernetes.io/projected/68c24a22-c9e1-4d60-bae0-819f1225272a-kube-api-access-cq729\") pod \"aws-ebs-csi-driver-node-c4r2x\" (UID: \"68c24a22-c9e1-4d60-bae0-819f1225272a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.817868 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.817786 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-csj2m" Jul 14 17:05:09.826682 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.826655 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-68j4h" Jul 14 17:05:09.827410 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.827384 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88367bbb_0442_4660_9d3f_b7cc31e51be1.slice/crio-976729ca7c1f18ca552cc040ef063dc30c6cb8d9e23f1f33b474b7fe6ac9b7bf WatchSource:0}: Error finding container 976729ca7c1f18ca552cc040ef063dc30c6cb8d9e23f1f33b474b7fe6ac9b7bf: Status 404 returned error can't find the container with id 976729ca7c1f18ca552cc040ef063dc30c6cb8d9e23f1f33b474b7fe6ac9b7bf Jul 14 17:05:09.834159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.834135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5xq9d" Jul 14 17:05:09.835542 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.835518 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd92ffff_5264_4b21_832d_88a3a9581098.slice/crio-473cd41fab865a991646ba30e4e052345bf3bf5bd79be76c75b0ec5980ba4880 WatchSource:0}: Error finding container 473cd41fab865a991646ba30e4e052345bf3bf5bd79be76c75b0ec5980ba4880: Status 404 returned error can't find the container with id 473cd41fab865a991646ba30e4e052345bf3bf5bd79be76c75b0ec5980ba4880 Jul 14 17:05:09.838374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.838353 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:09.842472 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.842454 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fnv55" Jul 14 17:05:09.842631 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.842567 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f3d5e8_a605_4a7d_8b81_5752720e1eb8.slice/crio-74116dd6b76e72b64aa774615be34f9872fd712a51133d6b3493ea009e696ed4 WatchSource:0}: Error finding container 74116dd6b76e72b64aa774615be34f9872fd712a51133d6b3493ea009e696ed4: Status 404 returned error can't find the container with id 74116dd6b76e72b64aa774615be34f9872fd712a51133d6b3493ea009e696ed4 Jul 14 17:05:09.847608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.847590 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-48gdt" Jul 14 17:05:09.848118 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.848091 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1090c57_e98d_4da2_bcc1_23cae48d366e.slice/crio-282727f7a0fbe96d0cdbb1be9b35f8361072fd00745b680869d73b87358f7583 WatchSource:0}: Error finding container 282727f7a0fbe96d0cdbb1be9b35f8361072fd00745b680869d73b87358f7583: Status 404 returned error can't find the container with id 282727f7a0fbe96d0cdbb1be9b35f8361072fd00745b680869d73b87358f7583 Jul 14 17:05:09.852226 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.852203 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df5604a_b419_4223_b152_acfd179dfb22.slice/crio-edd550930e8d4f684d9efb576684047c0878700ba2f6a80b96e00da66f789ea2 WatchSource:0}: Error finding container edd550930e8d4f684d9efb576684047c0878700ba2f6a80b96e00da66f789ea2: Status 404 returned error can't find the container with id edd550930e8d4f684d9efb576684047c0878700ba2f6a80b96e00da66f789ea2 Jul 14 17:05:09.853482 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.853452 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:09.857623 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.857596 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e6f316d_5b8a_4824_9f6e_dd7373442614.slice/crio-c3f726cadd9a3de94c0ac2a3e6545ca3d10afb19d0cea34dccecf32310a64a21 WatchSource:0}: Error finding container c3f726cadd9a3de94c0ac2a3e6545ca3d10afb19d0cea34dccecf32310a64a21: Status 404 returned error can't find the container with id c3f726cadd9a3de94c0ac2a3e6545ca3d10afb19d0cea34dccecf32310a64a21 Jul 14 17:05:09.858543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.858512 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4sw8m" Jul 14 17:05:09.863454 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:09.863432 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" Jul 14 17:05:09.864449 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.864427 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70960940_3298_4dee_842a_61b2d4494306.slice/crio-63311bfdc88b7b84e3e87d8cb227060f9384f5dfffdac28c7a8b55aca36be6c5 WatchSource:0}: Error finding container 63311bfdc88b7b84e3e87d8cb227060f9384f5dfffdac28c7a8b55aca36be6c5: Status 404 returned error can't find the container with id 63311bfdc88b7b84e3e87d8cb227060f9384f5dfffdac28c7a8b55aca36be6c5 Jul 14 17:05:09.868764 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.868742 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod425266e1_079d_46b8_8366_ce4b5651f269.slice/crio-0a46842188a50a03f1f041e91b54d948c8f62b934023722fcaf93d07a1903d5e WatchSource:0}: Error finding container 0a46842188a50a03f1f041e91b54d948c8f62b934023722fcaf93d07a1903d5e: Status 404 returned error can't find the container with id 0a46842188a50a03f1f041e91b54d948c8f62b934023722fcaf93d07a1903d5e Jul 14 17:05:09.874223 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:09.874199 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c24a22_c9e1_4d60_bae0_819f1225272a.slice/crio-1268a3b38871aa1b8d16ed0c6a5921cb1441a0e9215ca85387818918edf95029 WatchSource:0}: Error finding container 1268a3b38871aa1b8d16ed0c6a5921cb1441a0e9215ca85387818918edf95029: Status 404 returned error can't find the container with id 1268a3b38871aa1b8d16ed0c6a5921cb1441a0e9215ca85387818918edf95029 Jul 14 17:05:10.134433 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.134362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:10.134588 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.134523 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:10.134640 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.134599 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:11.134567679 +0000 UTC m=+4.657508455 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:10.235691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.235656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:10.235691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.235720 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:10.235990 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.235839 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:10.235990 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.235857 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:10.235990 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.235868 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:10.235990 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.235833 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:10.235990 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.235926 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:11.235907773 +0000 UTC m=+4.758848543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:10.236352 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.236024 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:11.235976272 +0000 UTC m=+4.758917038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:10.509838 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:10.509797 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd98126d381549eedd58b496d19efe54c.slice/crio-77fdc9273c698a37d2679f94abd2e41650a0054b41875e4fe3bac0cbbbb3abc8 WatchSource:0}: Error finding container 77fdc9273c698a37d2679f94abd2e41650a0054b41875e4fe3bac0cbbbb3abc8: Status 404 returned error can't find the container with id 77fdc9273c698a37d2679f94abd2e41650a0054b41875e4fe3bac0cbbbb3abc8 Jul 14 17:05:10.560930 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.560644 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:09 +0000 UTC" deadline="2027-12-30 18:40:02.316492984 +0000 UTC" Jul 14 17:05:10.560930 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.560933 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12817h34m51.755568374s" Jul 14 17:05:10.686594 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.686543 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:10.686755 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:10.686692 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:10.701604 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.701555 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5xq9d" event={"ID":"11f3d5e8-a605-4a7d-8b81-5752720e1eb8","Type":"ContainerStarted","Data":"74116dd6b76e72b64aa774615be34f9872fd712a51133d6b3493ea009e696ed4"} Jul 14 17:05:10.706936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.706902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-68j4h" event={"ID":"bd92ffff-5264-4b21-832d-88a3a9581098","Type":"ContainerStarted","Data":"473cd41fab865a991646ba30e4e052345bf3bf5bd79be76c75b0ec5980ba4880"} Jul 14 17:05:10.708877 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.708845 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-csj2m" event={"ID":"88367bbb-0442-4660-9d3f-b7cc31e51be1","Type":"ContainerStarted","Data":"976729ca7c1f18ca552cc040ef063dc30c6cb8d9e23f1f33b474b7fe6ac9b7bf"} Jul 14 17:05:10.712816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.712784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" event={"ID":"d98126d381549eedd58b496d19efe54c","Type":"ContainerStarted","Data":"77fdc9273c698a37d2679f94abd2e41650a0054b41875e4fe3bac0cbbbb3abc8"} Jul 14 17:05:10.726049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.725870 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" event={"ID":"68c24a22-c9e1-4d60-bae0-819f1225272a","Type":"ContainerStarted","Data":"1268a3b38871aa1b8d16ed0c6a5921cb1441a0e9215ca85387818918edf95029"} Jul 14 17:05:10.729972 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.729282 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4sw8m" event={"ID":"425266e1-079d-46b8-8366-ce4b5651f269","Type":"ContainerStarted","Data":"0a46842188a50a03f1f041e91b54d948c8f62b934023722fcaf93d07a1903d5e"} Jul 14 17:05:10.733812 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.733778 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"282727f7a0fbe96d0cdbb1be9b35f8361072fd00745b680869d73b87358f7583"} Jul 14 17:05:10.735640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.735610 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klggv" event={"ID":"70960940-3298-4dee-842a-61b2d4494306","Type":"ContainerStarted","Data":"63311bfdc88b7b84e3e87d8cb227060f9384f5dfffdac28c7a8b55aca36be6c5"} Jul 14 17:05:10.738125 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.738063 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerStarted","Data":"c3f726cadd9a3de94c0ac2a3e6545ca3d10afb19d0cea34dccecf32310a64a21"} Jul 14 17:05:10.741837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:10.741768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fnv55" event={"ID":"1df5604a-b419-4223-b152-acfd179dfb22","Type":"ContainerStarted","Data":"edd550930e8d4f684d9efb576684047c0878700ba2f6a80b96e00da66f789ea2"} Jul 14 17:05:11.142139 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.142023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:11.142613 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.142173 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:11.142613 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.142241 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:13.142220761 +0000 UTC m=+6.665161530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:11.242877 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.242825 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:11.243078 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.242911 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:11.243078 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243032 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:11.243208 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243095 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:13.243076568 +0000 UTC m=+6.766017325 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:11.243510 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243492 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:11.243626 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243519 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:11.243626 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243533 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:11.243626 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.243596 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:13.243564391 +0000 UTC m=+6.766505159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:11.561459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.561385 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:00:09 +0000 UTC" deadline="2028-03-05 03:27:40.976002253 +0000 UTC" Jul 14 17:05:11.561459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.561454 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14386h22m29.414552971s" Jul 14 17:05:11.686947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.686913 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:11.687124 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.687069 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:11.687523 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.687503 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:11.687651 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:11.687637 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:11.785472 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.785433 2574 generic.go:356] "Generic (PLEG): container finished" podID="1438194959089c40336a26a00fa74afd" containerID="673800061ff1d83b5c6dc167629a97569b41a43adfbf67401ad0066ef9d88049" exitCode=0 Jul 14 17:05:11.785659 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:11.785498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" event={"ID":"1438194959089c40336a26a00fa74afd","Type":"ContainerDied","Data":"673800061ff1d83b5c6dc167629a97569b41a43adfbf67401ad0066ef9d88049"} Jul 14 17:05:12.687824 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:12.687296 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:12.687824 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:12.687431 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:13.162138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:13.162034 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:13.162320 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.162192 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:13.162320 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.162258 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.162238474 +0000 UTC m=+10.685179231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:13.263334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:13.263295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:13.263519 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:13.263388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:13.263519 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263507 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:13.263700 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263584 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.263552725 +0000 UTC m=+10.786493482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:13.263700 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263683 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:13.263700 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263700 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:13.263871 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263712 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:13.263871 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.263747 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:17.263736276 +0000 UTC m=+10.786677033 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:13.691697 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:13.691181 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:13.691697 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:13.691184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:13.691697 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.691336 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:13.691697 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:13.691381 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:14.686639 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:14.686601 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:14.686840 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:14.686750 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:14.804668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:14.803915 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" event={"ID":"1438194959089c40336a26a00fa74afd","Type":"ContainerStarted","Data":"1187926eb7a8d85c1fb6ecf4b58d17ec72a7e2fd25c927c6f6a35b13027bfda6"} Jul 14 17:05:14.824671 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:14.824547 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-213.ec2.internal" podStartSLOduration=5.824524084 podStartE2EDuration="5.824524084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:14.823711649 +0000 UTC m=+8.346652459" watchObservedRunningTime="2026-07-14 17:05:14.824524084 +0000 UTC m=+8.347464858" Jul 14 17:05:15.687000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:15.686964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:15.687177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:15.686964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:15.687177 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:15.687118 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:15.687177 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:15.687165 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:16.687059 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:16.686884 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:16.687516 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:16.687108 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:17.196628 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:17.196562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:17.196823 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.196759 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:17.196887 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.196833 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:25.196810143 +0000 UTC m=+18.719750918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:17.297279 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:17.297236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:17.297427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:17.297357 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:17.297937 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.297505 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:17.297937 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.297586 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:25.297554125 +0000 UTC m=+18.820494891 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:17.297937 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.297920 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:17.297937 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.297944 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:17.298183 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.297957 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:17.298183 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.298005 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:25.2979902 +0000 UTC m=+18.820930965 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:17.687543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:17.687429 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:17.688092 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.687552 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:17.688092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:17.687599 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:17.688092 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:17.687668 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:18.686977 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:18.686939 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:18.687163 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:18.687058 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:19.686897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:19.686863 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:19.687511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:19.686913 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:19.687511 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:19.687023 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:19.687511 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:19.687128 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:20.686738 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:20.686700 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:20.686896 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:20.686835 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:21.687224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:21.687185 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:21.687224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:21.687217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:21.687690 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:21.687347 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:21.687690 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:21.687490 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:22.686849 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:22.686806 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:22.687033 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:22.686948 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:23.686950 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:23.686911 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:23.687386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:23.686965 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:23.687386 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:23.687058 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:23.687386 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:23.687198 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:24.686730 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:24.686681 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:24.686949 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:24.686812 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:25.259427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:25.259387 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:25.259925 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.259556 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:25.259925 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.259649 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:41.259631648 +0000 UTC m=+34.782572415 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:25.359852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:25.359814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:25.360041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:25.359879 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:25.360041 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.359987 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:25.360041 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.359999 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:25.360041 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.360024 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:25.360041 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.360039 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:25.360293 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.360054 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:41.360034659 +0000 UTC m=+34.882975409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:25.360293 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.360090 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:05:41.360076431 +0000 UTC m=+34.883017182 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:25.686619 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:25.686517 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:25.686782 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.686660 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:25.686782 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:25.686706 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:25.686889 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:25.686825 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:26.686937 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:26.686899 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:26.687342 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:26.687024 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:27.687618 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:27.687569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:27.688085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:27.687639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:27.688085 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:27.687697 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:27.688085 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:27.687764 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:28.687107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:28.687067 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:28.687365 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:28.687201 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:29.686309 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:29.686284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:29.686717 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:29.686324 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:29.686717 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:29.686413 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:29.686717 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:29.686496 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:29.853719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:29.853319 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-klggv" event={"ID":"70960940-3298-4dee-842a-61b2d4494306","Type":"ContainerStarted","Data":"a309e79b952f1ac6082307fd7b8c1e46fa21eb877722814d6d7e1d083f6d40b3"} Jul 14 17:05:29.862679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:29.861438 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerStarted","Data":"b8d488ed935d2a61ad15216a5d32f2f005fa29afad215fe819ccc02edbc38947"} Jul 14 17:05:29.899845 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:29.899790 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-klggv" podStartSLOduration=3.108305227 podStartE2EDuration="22.899764262s" totalImagesPullingTime="19.791459035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:29.874611796 +0000 UTC m=+23.397552570" watchObservedRunningTime="2026-07-14 17:05:29.899764262 +0000 UTC m=+23.422705035" Jul 14 17:05:30.687320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.687097 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:30.687999 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:30.687351 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:30.865507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.865466 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" event={"ID":"d98126d381549eedd58b496d19efe54c","Type":"ContainerStarted","Data":"8afc75a6945d39d8e01f05473bb3db355dafd81dece81c3777e65b95c89c7583"} Jul 14 17:05:30.866838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.866810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" event={"ID":"68c24a22-c9e1-4d60-bae0-819f1225272a","Type":"ContainerStarted","Data":"bca72ec103bc0c16ce337aa64799854b183845431513e4a34000397410ba8039"} Jul 14 17:05:30.868115 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.868090 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4sw8m" event={"ID":"425266e1-079d-46b8-8366-ce4b5651f269","Type":"ContainerStarted","Data":"69644116fc44036a2ccdf4768f8c3b500c715291efa752f80145ce89f757f257"} Jul 14 17:05:30.870659 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.870643 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:05:30.870954 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.870933 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1090c57-e98d-4da2-bcc1-23cae48d366e" containerID="1dcd39a96d6117d9cadc68db16143efdf3adbc85c2504dc2bc4c55a5d301a8e1" exitCode=1 Jul 14 17:05:30.871013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.870997 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"5d1d9f2ac8281604156e90583fbd570f00529d093ff26c796edfbdd2507482e4"} Jul 14 17:05:30.871067 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.871056 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"8588fbdfb84ab1a1a9c5d6512ee69dddfce76361c026db4f8cb0ff78598b2a20"} Jul 14 17:05:30.871119 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.871103 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"e3b0f5e12480ac28735ba25453e9265bd7e4359d3315597ff96faeb1e524b43c"} Jul 14 17:05:30.871169 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.871144 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"b8ee6faa27cdcbed610d23d2ff7a7dbb9e428f081d21271da058af07992a3e86"} Jul 14 17:05:30.871215 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.871178 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerDied","Data":"1dcd39a96d6117d9cadc68db16143efdf3adbc85c2504dc2bc4c55a5d301a8e1"} Jul 14 17:05:30.871255 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.871217 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"a51254cd9ab42077b4f558ed5636cefab9dbc3de0febf1aa80ee2628504d9d7a"} Jul 14 17:05:30.872373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.872357 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="b8d488ed935d2a61ad15216a5d32f2f005fa29afad215fe819ccc02edbc38947" exitCode=0 Jul 14 17:05:30.872457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.872418 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"b8d488ed935d2a61ad15216a5d32f2f005fa29afad215fe819ccc02edbc38947"} Jul 14 17:05:30.873838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.873813 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fnv55" event={"ID":"1df5604a-b419-4223-b152-acfd179dfb22","Type":"ContainerStarted","Data":"d1d8f71eebfc189a5dc260555d058b290ba50bb8574b62e4a94c5dbc2c7b79f1"} Jul 14 17:05:30.875186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.875168 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-68j4h" event={"ID":"bd92ffff-5264-4b21-832d-88a3a9581098","Type":"ContainerStarted","Data":"65ef75d545adf2bdec41cf6570701f99a7f9a72f44ff0aaa8f0c983aa19262a4"} Jul 14 17:05:30.876474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.876445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-csj2m" event={"ID":"88367bbb-0442-4660-9d3f-b7cc31e51be1","Type":"ContainerStarted","Data":"84661f275bf21cf1edb7b3332fa1e4e29ec39fbd9bd300fd36b7e6705eadd2a9"} Jul 14 17:05:30.882241 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.882206 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-213.ec2.internal" podStartSLOduration=21.882195751 podStartE2EDuration="21.882195751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:30.881831422 +0000 UTC m=+24.404772195" watchObservedRunningTime="2026-07-14 17:05:30.882195751 +0000 UTC m=+24.405136524" Jul 14 17:05:30.897126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.897079 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4sw8m" podStartSLOduration=4.109639193 podStartE2EDuration="23.897064237s" totalImagesPullingTime="19.787425044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:30.896831295 +0000 UTC m=+24.419772066" watchObservedRunningTime="2026-07-14 17:05:30.897064237 +0000 UTC m=+24.420005011" Jul 14 17:05:30.915171 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.915118 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fnv55" podStartSLOduration=4.122983198 podStartE2EDuration="23.91510428s" totalImagesPullingTime="19.792121082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:30.914646965 +0000 UTC m=+24.437587741" watchObservedRunningTime="2026-07-14 17:05:30.91510428 +0000 UTC m=+24.438045054" Jul 14 17:05:30.956895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.956787 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-csj2m" podStartSLOduration=4.127527736 podStartE2EDuration="23.956770602s" totalImagesPullingTime="19.829242866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:30.956304195 +0000 UTC m=+24.479244969" watchObservedRunningTime="2026-07-14 17:05:30.956770602 +0000 UTC m=+24.479711375" Jul 14 17:05:30.975505 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:30.975455 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-68j4h" podStartSLOduration=4.120329035 podStartE2EDuration="23.975439716s" totalImagesPullingTime="19.855110681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:30.975411131 +0000 UTC m=+24.498351901" watchObservedRunningTime="2026-07-14 17:05:30.975439716 +0000 UTC m=+24.498380489" Jul 14 17:05:31.284740 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.284708 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 17:05:31.589982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.589849 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:05:31.284733092Z","UUID":"df6416da-71bf-4044-86b7-e16ccaf7e38b","Handler":null,"Name":"","Endpoint":""} Jul 14 17:05:31.591272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.591247 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 17:05:31.591373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.591282 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 17:05:31.686422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.686387 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:31.686671 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.686387 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:31.686671 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:31.686510 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:31.686671 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:31.686554 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:31.881175 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.881091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5xq9d" event={"ID":"11f3d5e8-a605-4a7d-8b81-5752720e1eb8","Type":"ContainerStarted","Data":"c34337479b3343c9469b65bac7f3afb7a2d1667d83f23e5cfe01d8111db423e7"} Jul 14 17:05:31.883250 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.883214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" event={"ID":"68c24a22-c9e1-4d60-bae0-819f1225272a","Type":"ContainerStarted","Data":"810de33114ffd9cf72e86068691273827cc6adf3d33486995fe383fd87638527"} Jul 14 17:05:31.898720 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:31.898673 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5xq9d" podStartSLOduration=5.061566706 podStartE2EDuration="24.898657654s" totalImagesPullingTime="19.837090948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:31.898527178 +0000 UTC m=+25.421467952" watchObservedRunningTime="2026-07-14 17:05:31.898657654 +0000 UTC m=+25.421598430" Jul 14 17:05:32.581784 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.581747 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:32.687135 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.687096 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:32.687300 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:32.687222 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:32.887848 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.887755 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" event={"ID":"68c24a22-c9e1-4d60-bae0-819f1225272a","Type":"ContainerStarted","Data":"01690fa27e7cbc6d342f2b5492c8c36a042d86c53c7a0d03daea255d12807e7f"} Jul 14 17:05:32.891427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.891402 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:05:32.891839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.891811 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"ba6a89e3c1ad5bcd972d06885dcd2a4374a960b619fa50ca5b0015d6f4824a67"} Jul 14 17:05:32.919960 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.919866 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c4r2x" podStartSLOduration=3.7306690160000002 podStartE2EDuration="25.919846577s" totalImagesPullingTime="22.189177561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:32.919486161 +0000 UTC m=+26.442426937" watchObservedRunningTime="2026-07-14 17:05:32.919846577 +0000 UTC m=+26.442787349" Jul 14 17:05:32.957833 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.957799 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:32.958438 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:32.958419 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:33.687264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:33.687223 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:33.687587 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:33.687271 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:33.687587 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:33.687383 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:33.687587 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:33.687498 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:33.895387 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:33.895360 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-klggv" Jul 14 17:05:34.637479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:34.637399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:34.686471 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:34.686435 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:34.686657 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:34.686571 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:34.745818 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:34.745783 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:35.686941 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:35.686906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:35.687696 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:35.687056 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:35.687696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:35.687111 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:35.687696 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:35.687216 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:36.187163 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.187001 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:36.310880 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.310848 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:36.410871 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.410841 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:36.687182 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.687104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:36.687798 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:36.687235 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:36.903605 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.903564 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:05:36.903946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.903922 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"56adb93b5c815a26a29e3267024d9d620e619f5ce3dd407700e58037dc7b9ec9"} Jul 14 17:05:36.904194 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.904165 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:36.904389 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.904371 2574 scope.go:117] "RemoveContainer" containerID="1dcd39a96d6117d9cadc68db16143efdf3adbc85c2504dc2bc4c55a5d301a8e1" Jul 14 17:05:36.905995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.905967 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="a60ff16932e19da1ce8be8c77d26ce877b1203622b2818cc7aabbc20e5e7c56e" exitCode=0 Jul 14 17:05:36.906085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.906025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"a60ff16932e19da1ce8be8c77d26ce877b1203622b2818cc7aabbc20e5e7c56e"} Jul 14 17:05:36.921188 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:36.921162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:37.687474 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.687438 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:37.687996 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:37.687517 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:37.687996 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.687617 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:37.688122 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:37.688052 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:37.911366 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.911341 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:05:37.911695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.911671 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" event={"ID":"e1090c57-e98d-4da2-bcc1-23cae48d366e","Type":"ContainerStarted","Data":"d258c66deb2877c2a15da42fcf88fc6198425d8d6c5ce6c56cabd3477b64faf2"} Jul 14 17:05:37.911856 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.911841 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:37.911929 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.911915 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:37.927233 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.927202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:05:37.948968 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:37.948871 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" podStartSLOduration=10.953797631 podStartE2EDuration="30.948856778s" totalImagesPullingTime="19.995059147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:37.94745891 +0000 UTC m=+31.470399684" watchObservedRunningTime="2026-07-14 17:05:37.948856778 +0000 UTC m=+31.471797550" Jul 14 17:05:38.263207 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.263114 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nd78m"] Jul 14 17:05:38.263370 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.263250 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:38.263370 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:38.263357 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:38.263883 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.263863 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bmf4s"] Jul 14 17:05:38.263971 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.263964 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:38.264087 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:38.264066 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:38.264486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.264467 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-46tkm"] Jul 14 17:05:38.264561 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.264550 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:38.264647 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:38.264631 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:38.276487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.276460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:38.356836 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.356802 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:38.767558 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.767529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:38.868305 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:38.868263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:39.686344 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.686317 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:39.686514 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.686320 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:39.686514 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:39.686418 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:39.686514 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.686428 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:39.686514 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:39.686497 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:39.686735 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:39.686549 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:39.903536 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.903501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:39.919037 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.919003 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="9ba7430eae00e3be869e2f2dfd0ce4ab7e3ae912eb30cd271f74e04c2cda1e90" exitCode=0 Jul 14 17:05:39.919193 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.919092 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"9ba7430eae00e3be869e2f2dfd0ce4ab7e3ae912eb30cd271f74e04c2cda1e90"} Jul 14 17:05:39.968105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:39.968074 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:40.924128 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:40.924095 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="2a50d030468e5ddd2246c5812a8c618393ca9ac83d386371a1f041d0d424730d" exitCode=0 Jul 14 17:05:40.924524 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:40.924141 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"2a50d030468e5ddd2246c5812a8c618393ca9ac83d386371a1f041d0d424730d"} Jul 14 17:05:41.280760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.280730 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:41.280910 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.280868 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:41.280952 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.280929 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs podName:7ceb8907-d175-4f86-957c-e924c6093676 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:13.280914504 +0000 UTC m=+66.803855257 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs") pod "network-metrics-daemon-bmf4s" (UID: "7ceb8907-d175-4f86-957c-e924c6093676") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:05:41.381373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.381335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:41.381529 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.381394 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:41.381529 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381493 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:41.381529 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381503 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:05:41.381529 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381526 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:05:41.381674 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381536 2574 projected.go:196] Error preparing data for projected volume kube-api-access-xm6ht for pod openshift-network-diagnostics/network-check-target-nd78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:41.381674 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381547 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret podName:dba23291-95f1-4b81-9c35-80a0955c5935 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:13.381532108 +0000 UTC m=+66.904472862 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret") pod "global-pull-secret-syncer-46tkm" (UID: "dba23291-95f1-4b81-9c35-80a0955c5935") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:05:41.381674 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.381591 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht podName:46cd507f-65d2-479a-9df4-ecdce792350c nodeName:}" failed. No retries permitted until 2026-07-14 17:06:13.381563895 +0000 UTC m=+66.904504646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xm6ht" (UniqueName: "kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht") pod "network-check-target-nd78m" (UID: "46cd507f-65d2-479a-9df4-ecdce792350c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:05:41.399674 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.399638 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:41.554760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.554680 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:41.686492 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.686456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:41.686707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.686456 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:41.686707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:41.686618 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:41.686707 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.686621 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bmf4s" podUID="7ceb8907-d175-4f86-957c-e924c6093676" Jul 14 17:05:41.686894 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.686717 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nd78m" podUID="46cd507f-65d2-479a-9df4-ecdce792350c" Jul 14 17:05:41.686894 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:41.686812 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-46tkm" podUID="dba23291-95f1-4b81-9c35-80a0955c5935" Jul 14 17:05:42.686413 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:42.686353 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:42.789466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:42.789421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:43.480251 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.480170 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-213.ec2.internal" event="NodeReady" Jul 14 17:05:43.480387 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.480336 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 17:05:43.530295 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.530242 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:05:43.532971 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.532946 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x9nmv"] Jul 14 17:05:43.533107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.533078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.534859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.534838 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kg8cw"] Jul 14 17:05:43.534993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.534938 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.535559 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.535535 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 17:05:43.535792 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.535772 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 17:05:43.535879 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.535851 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 17:05:43.536020 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.536003 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-888d9\"" Jul 14 17:05:43.537116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.536792 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.538287 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.538267 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 17:05:43.538287 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.538279 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vhkzd\"" Jul 14 17:05:43.538431 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.538288 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 17:05:43.538700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.538679 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 17:05:43.541206 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.541184 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8v8vb\"" Jul 14 17:05:43.541308 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.541278 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 17:05:43.541379 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.541358 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 17:05:43.542513 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.542492 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x9nmv"] Jul 14 17:05:43.543181 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.542976 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 17:05:43.545422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.545394 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kg8cw"] Jul 14 17:05:43.549743 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.549718 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:05:43.671365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.671324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:43.687379 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.687341 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:05:43.687840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.687341 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:05:43.687840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.687341 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:05:43.691067 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691042 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:05:43.691299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691282 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\"" Jul 14 17:05:43.691414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691330 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:05:43.691461 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691447 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:05:43.691560 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691546 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:05:43.691707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.691689 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\"" Jul 14 17:05:43.697340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697315 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697379 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697409 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697435 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697598 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-config-volume\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.697598 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697697 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697702 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.697756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697730 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24fa006d-f6dc-4f46-83b1-cee1d2e66972-cert\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.697852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697764 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx6ld\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.697852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-tmp-dir\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.697852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfbmm\" (UniqueName: \"kubernetes.io/projected/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-kube-api-access-vfbmm\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.697852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697841 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgr5v\" (UniqueName: \"kubernetes.io/projected/24fa006d-f6dc-4f46-83b1-cee1d2e66972-kube-api-access-kgr5v\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.698005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.697899 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799034 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-config-volume\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799159 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799282 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24fa006d-f6dc-4f46-83b1-cee1d2e66972-cert\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799322 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx6ld\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799355 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-tmp-dir\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799380 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfbmm\" (UniqueName: \"kubernetes.io/projected/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-kube-api-access-vfbmm\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799411 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgr5v\" (UniqueName: \"kubernetes.io/projected/24fa006d-f6dc-4f46-83b1-cee1d2e66972-kube-api-access-kgr5v\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799454 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-config-volume\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:43.799612 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:43.799629 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr: secret "image-registry-tls" not found Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:43.799692 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls podName:2592ee91-ae20-4852-ad0a-03b6c3a177eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:44.299674692 +0000 UTC m=+37.822615447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls") pod "image-registry-7cfb8c4c5d-f9nhr" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb") : secret "image-registry-tls" not found Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:43.800618 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:43.800674 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls podName:4d8fd9ff-d18b-46e4-a3d5-3da9c997b619 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:44.300651431 +0000 UTC m=+37.823592183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls") pod "dns-default-kg8cw" (UID: "4d8fd9ff-d18b-46e4-a3d5-3da9c997b619") : secret "dns-default-metrics-tls" not found Jul 14 17:05:43.801330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.799463 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801708 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-tmp-dir\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.801954 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.802201 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.802611 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.802333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.806254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.806207 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24fa006d-f6dc-4f46-83b1-cee1d2e66972-cert\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.806254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.806239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.806442 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.806281 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.810534 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.810497 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgr5v\" (UniqueName: \"kubernetes.io/projected/24fa006d-f6dc-4f46-83b1-cee1d2e66972-kube-api-access-kgr5v\") pod \"ingress-canary-x9nmv\" (UID: \"24fa006d-f6dc-4f46-83b1-cee1d2e66972\") " pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:43.810668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.810598 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx6ld\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.811080 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.811057 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:43.811504 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.811484 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfbmm\" (UniqueName: \"kubernetes.io/projected/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-kube-api-access-vfbmm\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:43.857771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:43.857732 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x9nmv" Jul 14 17:05:44.207718 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.207473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:44.304783 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.304742 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:44.304967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.304814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:44.304967 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:44.304885 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:44.304967 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:44.304904 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr: secret "image-registry-tls" not found Jul 14 17:05:44.304967 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:44.304927 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:44.305170 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:44.304972 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls podName:2592ee91-ae20-4852-ad0a-03b6c3a177eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.304951941 +0000 UTC m=+38.827892712 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls") pod "image-registry-7cfb8c4c5d-f9nhr" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb") : secret "image-registry-tls" not found Jul 14 17:05:44.305170 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:44.304986 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls podName:4d8fd9ff-d18b-46e4-a3d5-3da9c997b619 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.30497987 +0000 UTC m=+38.827920620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls") pod "dns-default-kg8cw" (UID: "4d8fd9ff-d18b-46e4-a3d5-3da9c997b619") : secret "dns-default-metrics-tls" not found Jul 14 17:05:44.783739 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.783692 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:44.824767 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.824733 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z"] Jul 14 17:05:44.827088 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.827053 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" Jul 14 17:05:44.828257 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.828237 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jkwln"] Jul 14 17:05:44.830272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.830247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.830847 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.830823 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-rsjxd\"" Jul 14 17:05:44.832198 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.832169 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 17:05:44.832304 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.832171 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 17:05:44.836845 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.836405 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 17:05:44.837478 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.837287 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 17:05:44.837478 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.837320 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6x48r\"" Jul 14 17:05:44.837478 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.837361 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 17:05:44.837478 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.837288 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 17:05:44.838553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.838527 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z"] Jul 14 17:05:44.844867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.844843 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkwln"] Jul 14 17:05:44.875234 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.875194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:44.909495 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909454 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.909677 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909506 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e11e13b8-944b-47ed-8c06-1f8082cc3830-crio-socket\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.909677 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e11e13b8-944b-47ed-8c06-1f8082cc3830-data-volume\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.909677 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909561 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.909814 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909681 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trrs6\" (UniqueName: \"kubernetes.io/projected/32bb29f2-1a48-4524-b6d8-3fd711ee6fb3-kube-api-access-trrs6\") pod \"migrator-7f59b54f6c-g9z2z\" (UID: \"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" Jul 14 17:05:44.909814 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.909717 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzjhg\" (UniqueName: \"kubernetes.io/projected/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-api-access-rzjhg\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:44.983880 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:44.983849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzjhg\" (UniqueName: \"kubernetes.io/projected/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-api-access-rzjhg\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016252 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016286 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e11e13b8-944b-47ed-8c06-1f8082cc3830-crio-socket\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016313 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e11e13b8-944b-47ed-8c06-1f8082cc3830-data-volume\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.016405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trrs6\" (UniqueName: \"kubernetes.io/projected/32bb29f2-1a48-4524-b6d8-3fd711ee6fb3-kube-api-access-trrs6\") pod \"migrator-7f59b54f6c-g9z2z\" (UID: \"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.016447 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.016523 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.516503887 +0000 UTC m=+39.039444659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : configmap "kube-rbac-proxy" not found Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.017011 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e11e13b8-944b-47ed-8c06-1f8082cc3830-data-volume\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.017042 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.017142 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.017102 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:45.517087364 +0000 UTC m=+39.040028114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.017855 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.017208 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e11e13b8-944b-47ed-8c06-1f8082cc3830-crio-socket\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.031097 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.031061 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trrs6\" (UniqueName: \"kubernetes.io/projected/32bb29f2-1a48-4524-b6d8-3fd711ee6fb3-kube-api-access-trrs6\") pod \"migrator-7f59b54f6c-g9z2z\" (UID: \"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" Jul 14 17:05:45.031632 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.031607 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzjhg\" (UniqueName: \"kubernetes.io/projected/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-api-access-rzjhg\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.089417 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.089351 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x9nmv"] Jul 14 17:05:45.093325 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:45.093299 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24fa006d_f6dc_4f46_83b1_cee1d2e66972.slice/crio-f078de5d44802042917a32771878ab96a9828641116f8576077e4339f27be3a7 WatchSource:0}: Error finding container f078de5d44802042917a32771878ab96a9828641116f8576077e4339f27be3a7: Status 404 returned error can't find the container with id f078de5d44802042917a32771878ab96a9828641116f8576077e4339f27be3a7 Jul 14 17:05:45.139801 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.139762 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" Jul 14 17:05:45.264728 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.264698 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z"] Jul 14 17:05:45.269874 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:45.269841 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bb29f2_1a48_4524_b6d8_3fd711ee6fb3.slice/crio-6bc0b04e5ac0630a1ca5c36f2abf558e17c78d6a7a1a7acb1f874c549521c83f WatchSource:0}: Error finding container 6bc0b04e5ac0630a1ca5c36f2abf558e17c78d6a7a1a7acb1f874c549521c83f: Status 404 returned error can't find the container with id 6bc0b04e5ac0630a1ca5c36f2abf558e17c78d6a7a1a7acb1f874c549521c83f Jul 14 17:05:45.318189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.318159 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:45.318381 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.318216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:45.318381 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.318325 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:45.318381 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.318354 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr: secret "image-registry-tls" not found Jul 14 17:05:45.318381 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.318358 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:45.318517 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.318418 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls podName:2592ee91-ae20-4852-ad0a-03b6c3a177eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:47.318402961 +0000 UTC m=+40.841343712 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls") pod "image-registry-7cfb8c4c5d-f9nhr" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb") : secret "image-registry-tls" not found Jul 14 17:05:45.318517 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.318433 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls podName:4d8fd9ff-d18b-46e4-a3d5-3da9c997b619 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:47.318427546 +0000 UTC m=+40.841368297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls") pod "dns-default-kg8cw" (UID: "4d8fd9ff-d18b-46e4-a3d5-3da9c997b619") : secret "dns-default-metrics-tls" not found Jul 14 17:05:45.519157 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.518959 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.519344 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.519165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:45.519344 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.519102 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:45.519344 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.519272 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:46.519255999 +0000 UTC m=+40.042196750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : configmap "kube-rbac-proxy" not found Jul 14 17:05:45.519344 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.519304 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.519473 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:45.519351 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:46.519340141 +0000 UTC m=+40.042280892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:45.849930 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.849888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:45.939389 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.939309 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" event={"ID":"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3","Type":"ContainerStarted","Data":"6bc0b04e5ac0630a1ca5c36f2abf558e17c78d6a7a1a7acb1f874c549521c83f"} Jul 14 17:05:45.940837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.940744 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x9nmv" event={"ID":"24fa006d-f6dc-4f46-83b1-cee1d2e66972","Type":"ContainerStarted","Data":"f078de5d44802042917a32771878ab96a9828641116f8576077e4339f27be3a7"} Jul 14 17:05:45.944725 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.944692 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="d1d60ba5f20690f0cdefe591e0d6ef68ff191a889eb10798e0dc7137d347e13e" exitCode=0 Jul 14 17:05:45.944851 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.944768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"d1d60ba5f20690f0cdefe591e0d6ef68ff191a889eb10798e0dc7137d347e13e"} Jul 14 17:05:45.953383 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:45.953332 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:46.088681 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.088620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:46.188223 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.188087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:46.460805 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.460722 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-csj2m_88367bbb-0442-4660-9d3f-b7cc31e51be1/dns-node-resolver/0.log" Jul 14 17:05:46.529315 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.529274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:46.529500 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.529335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:46.529500 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:46.529378 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:46.529500 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:46.529467 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:48.529433911 +0000 UTC m=+42.052374689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : configmap "kube-rbac-proxy" not found Jul 14 17:05:46.529500 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:46.529487 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:46.529677 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:46.529548 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:48.529529666 +0000 UTC m=+42.052470438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:46.547419 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.547380 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:46.950930 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.950894 2574 generic.go:356] "Generic (PLEG): container finished" podID="3e6f316d-5b8a-4824-9f6e-dd7373442614" containerID="0c9a9f3b358d5058ccbca4c804fbe3419b292c9f6c0cc3e7f138a9c7d7d72d80" exitCode=0 Jul 14 17:05:46.951489 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:46.950943 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerDied","Data":"0c9a9f3b358d5058ccbca4c804fbe3419b292c9f6c0cc3e7f138a9c7d7d72d80"} Jul 14 17:05:47.337084 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:47.337046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:47.337260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:47.337116 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:47.337260 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:47.337234 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:47.337260 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:47.337235 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:47.337260 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:47.337262 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr: secret "image-registry-tls" not found Jul 14 17:05:47.337487 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:47.337297 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls podName:4d8fd9ff-d18b-46e4-a3d5-3da9c997b619 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:51.337281283 +0000 UTC m=+44.860222036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls") pod "dns-default-kg8cw" (UID: "4d8fd9ff-d18b-46e4-a3d5-3da9c997b619") : secret "dns-default-metrics-tls" not found Jul 14 17:05:47.337487 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:47.337327 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls podName:2592ee91-ae20-4852-ad0a-03b6c3a177eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:51.33730979 +0000 UTC m=+44.860250567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls") pod "image-registry-7cfb8c4c5d-f9nhr" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb") : secret "image-registry-tls" not found Jul 14 17:05:47.568112 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:47.568074 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:47.659193 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:47.659167 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4sw8m_425266e1-079d-46b8-8366-ce4b5651f269/node-ca/0.log" Jul 14 17:05:48.148611 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.148546 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-922bg"] Jul 14 17:05:48.164486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.164342 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-922bg"] Jul 14 17:05:48.164486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.164430 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.167252 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.167227 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 17:05:48.167383 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.167226 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 17:05:48.168256 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.168235 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jw4xw\"" Jul 14 17:05:48.168374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.168339 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 17:05:48.168779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.168238 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 17:05:48.255598 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.255385 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:48.345885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.345848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.345885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.345889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bpf4\" (UniqueName: \"kubernetes.io/projected/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-kube-api-access-8bpf4\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.346086 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.345980 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-key\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.355733 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.355697 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:48.447154 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.447120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.447154 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.447166 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bpf4\" (UniqueName: \"kubernetes.io/projected/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-kube-api-access-8bpf4\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.447400 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.447249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-key\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.447860 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.447840 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-cabundle\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.451265 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.451243 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-signing-key\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.456911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.456880 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bpf4\" (UniqueName: \"kubernetes.io/projected/e35ac47b-ad72-48ed-a15b-57ed3f3894d2-kube-api-access-8bpf4\") pod \"service-ca-79f68b9f4f-922bg\" (UID: \"e35ac47b-ad72-48ed-a15b-57ed3f3894d2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.475875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.475836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" Jul 14 17:05:48.549675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.549644 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:48.549749 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.549699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:48.549806 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:48.549785 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 17:05:48.549861 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:48.549848 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:48.549910 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:48.549868 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.549846397 +0000 UTC m=+46.072787170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : configmap "kube-rbac-proxy" not found Jul 14 17:05:48.549910 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:48.549906 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:52.549888531 +0000 UTC m=+46.072829287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:48.638403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.638314 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-922bg"] Jul 14 17:05:48.642116 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:48.642089 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35ac47b_ad72_48ed_a15b_57ed3f3894d2.slice/crio-f28caa3eed58b7920e18d7de45c7c68bf0c1f5426263e9d5982028c0f96b68e3 WatchSource:0}: Error finding container f28caa3eed58b7920e18d7de45c7c68bf0c1f5426263e9d5982028c0f96b68e3: Status 404 returned error can't find the container with id f28caa3eed58b7920e18d7de45c7c68bf0c1f5426263e9d5982028c0f96b68e3 Jul 14 17:05:48.678427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.678386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:48.746819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.746774 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:48.906013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.905928 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:48.958853 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.958815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-48gdt" event={"ID":"3e6f316d-5b8a-4824-9f6e-dd7373442614","Type":"ContainerStarted","Data":"4ed4e6d64f2e020d5407daae3876ef833b7f524ded84b783aa027a131640a7e8"} Jul 14 17:05:48.959995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.959971 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" event={"ID":"e35ac47b-ad72-48ed-a15b-57ed3f3894d2","Type":"ContainerStarted","Data":"f28caa3eed58b7920e18d7de45c7c68bf0c1f5426263e9d5982028c0f96b68e3"} Jul 14 17:05:48.961565 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.961543 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" event={"ID":"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3","Type":"ContainerStarted","Data":"a24374c9dfeddbebbda210b7fc14d405276d26d63893af7fd8ca346e9180b4fc"} Jul 14 17:05:48.961703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.961599 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" event={"ID":"32bb29f2-1a48-4524-b6d8-3fd711ee6fb3","Type":"ContainerStarted","Data":"227efb5134ab3b5c928f4c78cd5829419052ee379f14e402bf3c9e070c1f344c"} Jul 14 17:05:48.962900 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.962879 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x9nmv" event={"ID":"24fa006d-f6dc-4f46-83b1-cee1d2e66972","Type":"ContainerStarted","Data":"d82837f6c5e4e4a72d36ccc552048058b9b8789cf37bd9c5902c1ea74fddd0dd"} Jul 14 17:05:48.987475 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:48.987424 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-48gdt" podStartSLOduration=9.867624968 podStartE2EDuration="41.987407096s" totalImagesPullingTime="32.119782128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:48.985546887 +0000 UTC m=+42.508487659" watchObservedRunningTime="2026-07-14 17:05:48.987407096 +0000 UTC m=+42.510347868" Jul 14 17:05:49.004444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:49.004398 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-g9z2z" podStartSLOduration=2.239900644 podStartE2EDuration="5.004381785s" totalImagesPullingTime="2.764481141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:49.003162436 +0000 UTC m=+42.526103209" watchObservedRunningTime="2026-07-14 17:05:49.004381785 +0000 UTC m=+42.527322558" Jul 14 17:05:49.021469 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:49.021426 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x9nmv" podStartSLOduration=3.074186014 podStartE2EDuration="6.021411515s" totalImagesPullingTime="2.947225501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:49.020269295 +0000 UTC m=+42.543210070" watchObservedRunningTime="2026-07-14 17:05:49.021411515 +0000 UTC m=+42.544352282" Jul 14 17:05:50.020598 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:50.020540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:51.376017 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:51.375984 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:51.376048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:51.376140 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:51.376160 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr: secret "image-registry-tls" not found Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:51.376193 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:51.376213 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls podName:2592ee91-ae20-4852-ad0a-03b6c3a177eb nodeName:}" failed. No retries permitted until 2026-07-14 17:05:59.376193009 +0000 UTC m=+52.899133760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls") pod "image-registry-7cfb8c4c5d-f9nhr" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb") : secret "image-registry-tls" not found Jul 14 17:05:51.376421 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:51.376245 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls podName:4d8fd9ff-d18b-46e4-a3d5-3da9c997b619 nodeName:}" failed. No retries permitted until 2026-07-14 17:05:59.376233643 +0000 UTC m=+52.899174394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls") pod "dns-default-kg8cw" (UID: "4d8fd9ff-d18b-46e4-a3d5-3da9c997b619") : secret "dns-default-metrics-tls" not found Jul 14 17:05:51.971358 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:51.971317 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" event={"ID":"e35ac47b-ad72-48ed-a15b-57ed3f3894d2","Type":"ContainerStarted","Data":"09f957e64721a887fae19b762d1d3b5ab07c397a6af6ac3b677e0071a8cf8914"} Jul 14 17:05:51.990695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:51.990625 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-922bg" podStartSLOduration=1.381446464 podStartE2EDuration="3.990601416s" totalImagesPullingTime="2.609154952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:05:51.990116025 +0000 UTC m=+45.513056799" watchObservedRunningTime="2026-07-14 17:05:51.990601416 +0000 UTC m=+45.513542191" Jul 14 17:05:52.325009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.324919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:52.588502 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.588457 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:52.589976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.589634 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e11e13b8-944b-47ed-8c06-1f8082cc3830-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:52.590285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.590267 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:05:52.590612 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:52.590466 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 17:05:52.590764 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:05:52.590755 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls podName:e11e13b8-944b-47ed-8c06-1f8082cc3830 nodeName:}" failed. No retries permitted until 2026-07-14 17:06:00.590735805 +0000 UTC m=+54.113676572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jkwln" (UID: "e11e13b8-944b-47ed-8c06-1f8082cc3830") : secret "insights-runtime-extractor-tls" not found Jul 14 17:05:52.594864 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.594796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:52.697228 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.697192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:52.768673 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.768633 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:52.889143 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.889050 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:52.996617 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:52.996564 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:53.094149 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:53.094113 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:53.629212 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:53.629178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:54.341512 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:54.341479 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:55.279585 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:55.279534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:55.340393 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:55.340357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.040325 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.040291 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.289236 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.289195 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.397004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.396970 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.751489 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.751397 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.814177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.814138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:56.884448 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:56.884415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:57.729595 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:57.729529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:57.799794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:57.799755 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:58.093681 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:58.093633 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:58.485930 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:58.485842 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:59.442013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.441977 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:59.442494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.442024 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:59.444712 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.444687 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d8fd9ff-d18b-46e4-a3d5-3da9c997b619-metrics-tls\") pod \"dns-default-kg8cw\" (UID: \"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619\") " pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:59.444911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.444892 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"image-registry-7cfb8c4c5d-f9nhr\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:59.448675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.448654 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:05:59.462914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.462889 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kg8cw" Jul 14 17:05:59.560632 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.560539 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:59.600642 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.600609 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:05:59.604136 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:59.604102 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2592ee91_ae20_4852_ad0a_03b6c3a177eb.slice/crio-18cb62f11fe936ca859e1d4df849508019902fdfcee5feafb17e0629128762cd WatchSource:0}: Error finding container 18cb62f11fe936ca859e1d4df849508019902fdfcee5feafb17e0629128762cd: Status 404 returned error can't find the container with id 18cb62f11fe936ca859e1d4df849508019902fdfcee5feafb17e0629128762cd Jul 14 17:05:59.619586 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.619545 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kg8cw"] Jul 14 17:05:59.622367 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:05:59.622330 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8fd9ff_d18b_46e4_a3d5_3da9c997b619.slice/crio-5d2ddc4841a296f0e5b1ff678ce45ff44bcb007b5fe0d238a6515325beb2ca57 WatchSource:0}: Error finding container 5d2ddc4841a296f0e5b1ff678ce45ff44bcb007b5fe0d238a6515325beb2ca57: Status 404 returned error can't find the container with id 5d2ddc4841a296f0e5b1ff678ce45ff44bcb007b5fe0d238a6515325beb2ca57 Jul 14 17:05:59.628093 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.628066 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:59.704918 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.704892 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:05:59.992654 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.992557 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg8cw" event={"ID":"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619","Type":"ContainerStarted","Data":"5d2ddc4841a296f0e5b1ff678ce45ff44bcb007b5fe0d238a6515325beb2ca57"} Jul 14 17:05:59.994040 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.994013 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" event={"ID":"2592ee91-ae20-4852-ad0a-03b6c3a177eb","Type":"ContainerStarted","Data":"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263"} Jul 14 17:05:59.994180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.994049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" event={"ID":"2592ee91-ae20-4852-ad0a-03b6c3a177eb","Type":"ContainerStarted","Data":"18cb62f11fe936ca859e1d4df849508019902fdfcee5feafb17e0629128762cd"} Jul 14 17:05:59.994180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:05:59.994134 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:06:00.021675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.021620 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" podStartSLOduration=34.021602999 podStartE2EDuration="34.021602999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:00.020284293 +0000 UTC m=+53.543225066" watchObservedRunningTime="2026-07-14 17:06:00.021602999 +0000 UTC m=+53.544543771" Jul 14 17:06:00.504286 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.504245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:00.575595 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.575550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:00.653555 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.653521 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:06:00.655931 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.655900 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e11e13b8-944b-47ed-8c06-1f8082cc3830-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jkwln\" (UID: \"e11e13b8-944b-47ed-8c06-1f8082cc3830\") " pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:06:00.746920 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.746885 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jkwln" Jul 14 17:06:00.889244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.889185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:00.930001 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:00.929969 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jkwln"] Jul 14 17:06:00.934322 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:00.934285 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode11e13b8_944b_47ed_8c06_1f8082cc3830.slice/crio-440a61cf56f04b31ccc415abff266c9eb9844d5b266051902c5502aa9805dd2b WatchSource:0}: Error finding container 440a61cf56f04b31ccc415abff266c9eb9844d5b266051902c5502aa9805dd2b: Status 404 returned error can't find the container with id 440a61cf56f04b31ccc415abff266c9eb9844d5b266051902c5502aa9805dd2b Jul 14 17:06:01.002039 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:01.001997 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkwln" event={"ID":"e11e13b8-944b-47ed-8c06-1f8082cc3830","Type":"ContainerStarted","Data":"440a61cf56f04b31ccc415abff266c9eb9844d5b266051902c5502aa9805dd2b"} Jul 14 17:06:01.309253 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:01.309216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:01.492381 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:01.492344 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:01.636353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:01.636311 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:02.007327 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:02.007266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg8cw" event={"ID":"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619","Type":"ContainerStarted","Data":"8f10c0377818185c87e38d9b7a21e9765f1f7e5cc3d4f2b2638d5275bf30ac60"} Jul 14 17:06:02.093507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:02.093421 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:02.487423 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:02.487364 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:02.893629 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:02.893588 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:03.014587 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:03.014550 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kg8cw" event={"ID":"4d8fd9ff-d18b-46e4-a3d5-3da9c997b619","Type":"ContainerStarted","Data":"4f7745b451f425962c6647e30444c7e2ab1ca4d14408d27d24ed3bf11dbdc2c0"} Jul 14 17:06:03.035696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:03.035639 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kg8cw" podStartSLOduration=17.797560412 podStartE2EDuration="20.035616998s" totalImagesPullingTime="2.238056586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:03.03496837 +0000 UTC m=+56.557909142" watchObservedRunningTime="2026-07-14 17:06:03.035616998 +0000 UTC m=+56.558557755" Jul 14 17:06:03.083086 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:03.083027 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:03.293230 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:03.293139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:04.019276 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:04.019242 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kg8cw" Jul 14 17:06:06.026739 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:06.026697 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkwln" event={"ID":"e11e13b8-944b-47ed-8c06-1f8082cc3830","Type":"ContainerStarted","Data":"dfba8ef3c6d3c7d0aea59268842915bb7eed90748d7198a44af408894b704d35"} Jul 14 17:06:07.031369 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.031289 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkwln" event={"ID":"e11e13b8-944b-47ed-8c06-1f8082cc3830","Type":"ContainerStarted","Data":"a15ebffa1f40aea39298dcc80d7065d3a02ec0590d026fdda269aba1a6fafc58"} Jul 14 17:06:07.251315 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.251279 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv"] Jul 14 17:06:07.253271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.253252 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.256463 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.256434 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 17:06:07.256913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.256895 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 17:06:07.257984 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.257963 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 17:06:07.258116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.257963 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 17:06:07.265691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.265664 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv"] Jul 14 17:06:07.311162 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.311079 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57jzb\" (UniqueName: \"kubernetes.io/projected/e1e042e8-46e3-4137-ac5b-1890799c95cd-kube-api-access-57jzb\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.311320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.311277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1e042e8-46e3-4137-ac5b-1890799c95cd-tmp\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.311370 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.311332 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1e042e8-46e3-4137-ac5b-1890799c95cd-klusterlet-config\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.390455 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.390427 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:06:07.411732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.411699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1e042e8-46e3-4137-ac5b-1890799c95cd-tmp\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.411907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.411751 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1e042e8-46e3-4137-ac5b-1890799c95cd-klusterlet-config\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.411907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.411772 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57jzb\" (UniqueName: \"kubernetes.io/projected/e1e042e8-46e3-4137-ac5b-1890799c95cd-kube-api-access-57jzb\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.412169 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.412138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1e042e8-46e3-4137-ac5b-1890799c95cd-tmp\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.414658 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.414612 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e1e042e8-46e3-4137-ac5b-1890799c95cd-klusterlet-config\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.427355 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.427329 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57jzb\" (UniqueName: \"kubernetes.io/projected/e1e042e8-46e3-4137-ac5b-1890799c95cd-kube-api-access-57jzb\") pod \"klusterlet-addon-workmgr-6df88b8b4d-t86xv\" (UID: \"e1e042e8-46e3-4137-ac5b-1890799c95cd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.479854 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.479823 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8476656947-58b5p"] Jul 14 17:06:07.480016 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.479886 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:07.481892 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.481870 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.500031 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.500003 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8476656947-58b5p"] Jul 14 17:06:07.569026 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.568934 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 17:06:07.573999 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.573973 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:07.613661 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613630 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-bound-sa-token\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613818 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613695 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-installation-pull-secrets\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613818 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613783 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3658f22-9430-4dd4-b020-52dd9f408473-ca-trust-extracted\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613834 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-registry-certificates\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613854 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnqvl\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-kube-api-access-wnqvl\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613877 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-image-registry-private-configuration\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.613914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613904 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-trusted-ca\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.614119 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.613947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-registry-tls\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.622915 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.622832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:07.715233 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-registry-certificates\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715233 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnqvl\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-kube-api-access-wnqvl\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715255 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-image-registry-private-configuration\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-trusted-ca\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-registry-tls\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715452 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-bound-sa-token\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715823 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-installation-pull-secrets\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.715885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.715871 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3658f22-9430-4dd4-b020-52dd9f408473-ca-trust-extracted\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.716149 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.716131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-registry-certificates\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.716295 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.716273 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d3658f22-9430-4dd4-b020-52dd9f408473-ca-trust-extracted\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.716543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.716521 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d3658f22-9430-4dd4-b020-52dd9f408473-trusted-ca\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.718390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.718364 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-image-registry-private-configuration\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.718479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.718465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-registry-tls\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.718712 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.718693 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d3658f22-9430-4dd4-b020-52dd9f408473-installation-pull-secrets\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.724136 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.724116 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnqvl\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-kube-api-access-wnqvl\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.729873 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.729843 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:07.730221 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.730198 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv"] Jul 14 17:06:07.733341 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.733323 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3658f22-9430-4dd4-b020-52dd9f408473-bound-sa-token\") pod \"image-registry-8476656947-58b5p\" (UID: \"d3658f22-9430-4dd4-b020-52dd9f408473\") " pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.734863 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:07.734837 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e042e8_46e3_4137_ac5b_1890799c95cd.slice/crio-ec0784a357f35e4a73cea77a8c74135cb037ea817c1e29847c0412e8b1ffe0d0 WatchSource:0}: Error finding container ec0784a357f35e4a73cea77a8c74135cb037ea817c1e29847c0412e8b1ffe0d0: Status 404 returned error can't find the container with id ec0784a357f35e4a73cea77a8c74135cb037ea817c1e29847c0412e8b1ffe0d0 Jul 14 17:06:07.791467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.791425 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:07.903052 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.903000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:07.966726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:07.966698 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8476656947-58b5p"] Jul 14 17:06:07.968479 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:07.968437 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3658f22_9430_4dd4_b020_52dd9f408473.slice/crio-c63e52f7caf368317ac735467b64965bf9819233b407924980ecf977827502ab WatchSource:0}: Error finding container c63e52f7caf368317ac735467b64965bf9819233b407924980ecf977827502ab: Status 404 returned error can't find the container with id c63e52f7caf368317ac735467b64965bf9819233b407924980ecf977827502ab Jul 14 17:06:08.036469 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.036432 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8476656947-58b5p" event={"ID":"d3658f22-9430-4dd4-b020-52dd9f408473","Type":"ContainerStarted","Data":"c63e52f7caf368317ac735467b64965bf9819233b407924980ecf977827502ab"} Jul 14 17:06:08.037906 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.037861 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" event={"ID":"e1e042e8-46e3-4137-ac5b-1890799c95cd","Type":"ContainerStarted","Data":"ec0784a357f35e4a73cea77a8c74135cb037ea817c1e29847c0412e8b1ffe0d0"} Jul 14 17:06:08.446883 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.446831 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:08.653170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.653130 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-lp9p6"] Jul 14 17:06:08.668178 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.668134 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:08.691358 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.691327 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lp9p6"] Jul 14 17:06:08.691539 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.691461 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:08.694938 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.694913 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 17:06:08.694938 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.694937 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-924z6\"" Jul 14 17:06:08.695137 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.694920 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 17:06:08.740604 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.740556 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:08.826191 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.826146 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pvqw\" (UniqueName: \"kubernetes.io/projected/742b8fa1-9d7e-4e17-8fb1-5f211689026d-kube-api-access-2pvqw\") pod \"downloads-68c9cbb497-lp9p6\" (UID: \"742b8fa1-9d7e-4e17-8fb1-5f211689026d\") " pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:08.834249 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.834197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:08.926694 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.926652 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pvqw\" (UniqueName: \"kubernetes.io/projected/742b8fa1-9d7e-4e17-8fb1-5f211689026d-kube-api-access-2pvqw\") pod \"downloads-68c9cbb497-lp9p6\" (UID: \"742b8fa1-9d7e-4e17-8fb1-5f211689026d\") " pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:08.935192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.935158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:08.937700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:08.937673 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pvqw\" (UniqueName: \"kubernetes.io/projected/742b8fa1-9d7e-4e17-8fb1-5f211689026d-kube-api-access-2pvqw\") pod \"downloads-68c9cbb497-lp9p6\" (UID: \"742b8fa1-9d7e-4e17-8fb1-5f211689026d\") " pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:09.002439 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.002361 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:09.043547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.043487 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8476656947-58b5p" event={"ID":"d3658f22-9430-4dd4-b020-52dd9f408473","Type":"ContainerStarted","Data":"267970c06cdb2a039383ffe538a9e6079ed634daf88bf57091de7c7d9cd227fa"} Jul 14 17:06:09.043982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.043652 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:09.046271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.046234 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.067535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.067462 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8476656947-58b5p" podStartSLOduration=2.067438622 podStartE2EDuration="2.067438622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:09.066347587 +0000 UTC m=+62.589288357" watchObservedRunningTime="2026-07-14 17:06:09.067438622 +0000 UTC m=+62.590379390" Jul 14 17:06:09.146213 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.146178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.302958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.302878 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.531071 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.531044 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lp9p6"] Jul 14 17:06:09.534502 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:09.534473 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742b8fa1_9d7e_4e17_8fb1_5f211689026d.slice/crio-8a01bdd6dcaef94825ef4c133c47d1c3f7ac2b7eef38db14264f7cd001dbf290 WatchSource:0}: Error finding container 8a01bdd6dcaef94825ef4c133c47d1c3f7ac2b7eef38db14264f7cd001dbf290: Status 404 returned error can't find the container with id 8a01bdd6dcaef94825ef4c133c47d1c3f7ac2b7eef38db14264f7cd001dbf290 Jul 14 17:06:09.579917 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.579885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.726214 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.726181 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.792418 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.792326 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:09.933712 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:09.932923 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tj95b" Jul 14 17:06:10.049035 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.048991 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lp9p6" event={"ID":"742b8fa1-9d7e-4e17-8fb1-5f211689026d","Type":"ContainerStarted","Data":"8a01bdd6dcaef94825ef4c133c47d1c3f7ac2b7eef38db14264f7cd001dbf290"} Jul 14 17:06:10.052681 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.052602 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jkwln" event={"ID":"e11e13b8-944b-47ed-8c06-1f8082cc3830","Type":"ContainerStarted","Data":"ccec6045c5489601f95c38584dfc4cf7251efb409a79d3df23135c3fabe69fe5"} Jul 14 17:06:10.090069 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.090005 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jkwln" podStartSLOduration=17.416997318 podStartE2EDuration="26.089982261s" totalImagesPullingTime="8.672984943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:10.089307369 +0000 UTC m=+63.612248144" watchObservedRunningTime="2026-07-14 17:06:10.089982261 +0000 UTC m=+63.612923036" Jul 14 17:06:10.103154 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.103061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:10.176162 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.176073 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:10.531975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.531939 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:10.630056 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.630018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:10.891832 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:10.891721 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:11.025604 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:11.025382 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kg8cw" Jul 14 17:06:11.411921 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:11.411881 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:11.482734 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:11.482663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:11.759778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:11.759693 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:13.369042 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.368996 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:06:13.372188 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.372156 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:06:13.382790 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.382761 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ceb8907-d175-4f86-957c-e924c6093676-metrics-certs\") pod \"network-metrics-daemon-bmf4s\" (UID: \"7ceb8907-d175-4f86-957c-e924c6093676\") " pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:06:13.415386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.415349 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f7wgm\"" Jul 14 17:06:13.421907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.421879 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bmf4s" Jul 14 17:06:13.469970 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.469939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:06:13.470156 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.470000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:06:13.473295 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.473263 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:06:13.474454 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.474408 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:06:13.483779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.483673 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:06:13.484397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.484330 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dba23291-95f1-4b81-9c35-80a0955c5935-original-pull-secret\") pod \"global-pull-secret-syncer-46tkm\" (UID: \"dba23291-95f1-4b81-9c35-80a0955c5935\") " pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:06:13.493109 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.493084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm6ht\" (UniqueName: \"kubernetes.io/projected/46cd507f-65d2-479a-9df4-ecdce792350c-kube-api-access-xm6ht\") pod \"network-check-target-nd78m\" (UID: \"46cd507f-65d2-479a-9df4-ecdce792350c\") " pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:06:13.570859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.570834 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bmf4s"] Jul 14 17:06:13.573214 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:13.573185 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ceb8907_d175_4f86_957c_e924c6093676.slice/crio-a6b0089e2c385565ab10e21e87aaa924a0c9b6e17303d8c444cebb9842593f47 WatchSource:0}: Error finding container a6b0089e2c385565ab10e21e87aaa924a0c9b6e17303d8c444cebb9842593f47: Status 404 returned error can't find the container with id a6b0089e2c385565ab10e21e87aaa924a0c9b6e17303d8c444cebb9842593f47 Jul 14 17:06:13.702608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.702510 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tv9jb\"" Jul 14 17:06:13.706095 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.706062 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-46tkm" Jul 14 17:06:13.709778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.709758 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:06:13.885101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.885068 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-46tkm"] Jul 14 17:06:13.888723 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:13.888695 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba23291_95f1_4b81_9c35_80a0955c5935.slice/crio-d4ebb1255d6cb9ded48e8d742931d484e763828a06cc9c6d041b39ee5d551b66 WatchSource:0}: Error finding container d4ebb1255d6cb9ded48e8d742931d484e763828a06cc9c6d041b39ee5d551b66: Status 404 returned error can't find the container with id d4ebb1255d6cb9ded48e8d742931d484e763828a06cc9c6d041b39ee5d551b66 Jul 14 17:06:13.901705 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:13.901666 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nd78m"] Jul 14 17:06:13.905177 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:13.905133 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46cd507f_65d2_479a_9df4_ecdce792350c.slice/crio-0b09fb93643a7844ad20fdaae0c0106b3393ec1316b44bc5eb20b1bdac2af40e WatchSource:0}: Error finding container 0b09fb93643a7844ad20fdaae0c0106b3393ec1316b44bc5eb20b1bdac2af40e: Status 404 returned error can't find the container with id 0b09fb93643a7844ad20fdaae0c0106b3393ec1316b44bc5eb20b1bdac2af40e Jul 14 17:06:14.067754 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.067662 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" event={"ID":"e1e042e8-46e3-4137-ac5b-1890799c95cd","Type":"ContainerStarted","Data":"2c9458de2c23b7a3c62f7e6c05f284f4c33ffe1b85b2ae371b51afcf14c00701"} Jul 14 17:06:14.067912 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.067869 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:14.068939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.068913 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-46tkm" event={"ID":"dba23291-95f1-4b81-9c35-80a0955c5935","Type":"ContainerStarted","Data":"d4ebb1255d6cb9ded48e8d742931d484e763828a06cc9c6d041b39ee5d551b66"} Jul 14 17:06:14.069836 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.069815 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" Jul 14 17:06:14.070041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.070022 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmf4s" event={"ID":"7ceb8907-d175-4f86-957c-e924c6093676","Type":"ContainerStarted","Data":"a6b0089e2c385565ab10e21e87aaa924a0c9b6e17303d8c444cebb9842593f47"} Jul 14 17:06:14.071012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.070993 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nd78m" event={"ID":"46cd507f-65d2-479a-9df4-ecdce792350c","Type":"ContainerStarted","Data":"0b09fb93643a7844ad20fdaae0c0106b3393ec1316b44bc5eb20b1bdac2af40e"} Jul 14 17:06:14.090898 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.090851 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6df88b8b4d-t86xv" podStartSLOduration=1.784667834 podStartE2EDuration="7.09083524s" totalImagesPullingTime="5.306167406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:14.090195317 +0000 UTC m=+67.613136089" watchObservedRunningTime="2026-07-14 17:06:14.09083524 +0000 UTC m=+67.613776024" Jul 14 17:06:14.304710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.304670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:14.989194 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:14.989155 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:15.218998 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:15.218916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:15.316526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:15.316443 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:15.430887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:15.430850 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.049107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.049046 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.084912 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.084347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmf4s" event={"ID":"7ceb8907-d175-4f86-957c-e924c6093676","Type":"ContainerStarted","Data":"9934135a098c4e14b3e76d938db7794644c1ab10eef9e09fa1bd09f48ba7579f"} Jul 14 17:06:16.084912 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.084457 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bmf4s" event={"ID":"7ceb8907-d175-4f86-957c-e924c6093676","Type":"ContainerStarted","Data":"cb91fc7341c9f352b4926d19b336c649b0b49b08a09046d06eee928fc9ac81a9"} Jul 14 17:06:16.110151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.110086 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bmf4s" podStartSLOduration=67.048334108 podStartE2EDuration="1m9.110065813s" totalImagesPullingTime="2.061731705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:16.11002379 +0000 UTC m=+69.632964555" watchObservedRunningTime="2026-07-14 17:06:16.110065813 +0000 UTC m=+69.633006584" Jul 14 17:06:16.138775 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.138723 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.203081 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.202903 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.274640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.274604 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.640357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.640239 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:16.731138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:16.731095 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:17.399550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:17.399318 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:06:18.164837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.164803 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:18.182127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.182057 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:18.182305 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.182182 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.190082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.190051 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 17:06:18.190244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.190088 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 17:06:18.191300 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.190408 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 17:06:18.193947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.193061 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 17:06:18.193947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.193216 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 17:06:18.193947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.193403 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jkhdp\"" Jul 14 17:06:18.279160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.279108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:18.310885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.310847 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.311082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.310912 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.311082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.311018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4kdj\" (UniqueName: \"kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.311082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.311054 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.311229 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.311082 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.311229 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.311152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.380608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.380556 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:18.412457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4kdj\" (UniqueName: \"kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.412457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412473 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412525 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.412653 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.413295 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.413732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.413444 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.414099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.414047 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.416158 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.416069 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.416370 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.416335 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.424907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.424877 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4kdj\" (UniqueName: \"kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj\") pod \"console-68679bb776-zqd8m\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:18.477123 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.477040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:18.496368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:18.496334 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:19.472016 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:19.471979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:19.787879 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:19.787803 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:19.791042 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:19.791010 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e06efe5_093a_48de_a48b_97ec7e9d1979.slice/crio-c36921fb601955e2c980f71b043a33936a40f958e58158a59791b0b77b960383 WatchSource:0}: Error finding container c36921fb601955e2c980f71b043a33936a40f958e58158a59791b0b77b960383: Status 404 returned error can't find the container with id c36921fb601955e2c980f71b043a33936a40f958e58158a59791b0b77b960383 Jul 14 17:06:20.102393 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.102358 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-46tkm" event={"ID":"dba23291-95f1-4b81-9c35-80a0955c5935","Type":"ContainerStarted","Data":"ea793fcf2eccf28625b9008cda3bfc4bf373dd88264dc64c0ccdb3db08d72109"} Jul 14 17:06:20.103906 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.103880 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68679bb776-zqd8m" event={"ID":"9e06efe5-093a-48de-a48b-97ec7e9d1979","Type":"ContainerStarted","Data":"c36921fb601955e2c980f71b043a33936a40f958e58158a59791b0b77b960383"} Jul 14 17:06:20.105782 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.105723 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nd78m" event={"ID":"46cd507f-65d2-479a-9df4-ecdce792350c","Type":"ContainerStarted","Data":"827355d579e8e11dfb7a5ef1b217d0a20aa13f051c70774e2959375b15312524"} Jul 14 17:06:20.105952 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.105810 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:06:20.131458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.131411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:20.142467 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.142409 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nd78m" podStartSLOduration=67.405492258 podStartE2EDuration="1m13.142387249s" totalImagesPullingTime="5.736894991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:20.141273283 +0000 UTC m=+73.664214052" watchObservedRunningTime="2026-07-14 17:06:20.142387249 +0000 UTC m=+73.665328022" Jul 14 17:06:20.142939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.142876 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-46tkm" podStartSLOduration=67.369275335 podStartE2EDuration="1m13.142860877s" totalImagesPullingTime="5.773585542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:20.121339095 +0000 UTC m=+73.644279869" watchObservedRunningTime="2026-07-14 17:06:20.142860877 +0000 UTC m=+73.665801650" Jul 14 17:06:20.146726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.146700 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:06:20.148686 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.148664 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.156533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.156475 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 17:06:20.162120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.162087 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:06:20.229961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.229921 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2lbl\" (UniqueName: \"kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.229981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.230029 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.230076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.230099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.230151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.230479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.230284 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.231993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.231901 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:20.331494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331570 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331651 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2lbl\" (UniqueName: \"kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.331752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.331719 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.332768 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.332729 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.336947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.333103 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.336947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.333294 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.336947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.333645 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.336947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.334830 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.336947 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.334872 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.342308 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.342282 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2lbl\" (UniqueName: \"kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl\") pod \"console-7ddcdf4c6b-svr4x\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.378101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.378015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:20.459711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.459672 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:20.615254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:20.615141 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:06:20.618762 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:20.618698 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae4e9d8_8cef_444a_9034_87771d159372.slice/crio-4c5bed6058ade8b35a94591d4c8cd5f06b8018ecb71f4f666df3687589fa57e1 WatchSource:0}: Error finding container 4c5bed6058ade8b35a94591d4c8cd5f06b8018ecb71f4f666df3687589fa57e1: Status 404 returned error can't find the container with id 4c5bed6058ade8b35a94591d4c8cd5f06b8018ecb71f4f666df3687589fa57e1 Jul 14 17:06:21.111287 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:21.111242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcdf4c6b-svr4x" event={"ID":"1ae4e9d8-8cef-444a-9034-87771d159372","Type":"ContainerStarted","Data":"4c5bed6058ade8b35a94591d4c8cd5f06b8018ecb71f4f666df3687589fa57e1"} Jul 14 17:06:21.383817 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:21.383687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:22.099408 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:22.099322 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:22.197809 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:22.197772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:23.356968 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:23.356823 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:23.824209 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:23.824169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:24.124822 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:24.124784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68679bb776-zqd8m" event={"ID":"9e06efe5-093a-48de-a48b-97ec7e9d1979","Type":"ContainerStarted","Data":"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1"} Jul 14 17:06:24.126563 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:24.126537 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcdf4c6b-svr4x" event={"ID":"1ae4e9d8-8cef-444a-9034-87771d159372","Type":"ContainerStarted","Data":"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f"} Jul 14 17:06:24.160279 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:24.160221 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68679bb776-zqd8m" podStartSLOduration=2.366030998 podStartE2EDuration="6.160198997s" totalImagesPullingTime="3.794167999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:24.158622371 +0000 UTC m=+77.681563145" watchObservedRunningTime="2026-07-14 17:06:24.160198997 +0000 UTC m=+77.683139769" Jul 14 17:06:24.199735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:24.199404 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ddcdf4c6b-svr4x" podStartSLOduration=1.237036842 podStartE2EDuration="4.199380286s" totalImagesPullingTime="2.962343444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:24.198347415 +0000 UTC m=+77.721288191" watchObservedRunningTime="2026-07-14 17:06:24.199380286 +0000 UTC m=+77.722321051" Jul 14 17:06:25.650390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:25.650343 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:25.906238 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:25.906110 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:26.010297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.010255 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-dkwzc"] Jul 14 17:06:26.013551 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.013510 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.028407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.028357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:26.029711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.029689 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 17:06:26.029857 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.029834 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 17:06:26.030906 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.030887 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 17:06:26.031022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.030895 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 17:06:26.031022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.030979 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 17:06:26.031022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.030893 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pxj4z\"" Jul 14 17:06:26.031236 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.030887 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 17:06:26.139306 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.139260 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:26.183200 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183116 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-wtmp\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183200 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-sys\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183232 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-tls\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183302 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183377 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-metrics-client-ca\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-root\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183470 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-textfile\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.183668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.183504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lln6w\" (UniqueName: \"kubernetes.io/projected/3a860158-6ffe-4345-8c2a-e1661cb29d76-kube-api-access-lln6w\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.235505 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.235468 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:26.284535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284502 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-tls\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-metrics-client-ca\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-root\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284726 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284703 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-root\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284924 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284902 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-textfile\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.284973 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.284955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lln6w\" (UniqueName: \"kubernetes.io/projected/3a860158-6ffe-4345-8c2a-e1661cb29d76-kube-api-access-lln6w\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285031 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-wtmp\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285089 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-sys\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285101 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285303 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-metrics-client-ca\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285652 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285507 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-textfile\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285652 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285589 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-wtmp\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285652 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285604 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3a860158-6ffe-4345-8c2a-e1661cb29d76-sys\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.285881 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.285669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-accelerators-collector-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.287449 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.287425 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.287600 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.287563 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3a860158-6ffe-4345-8c2a-e1661cb29d76-node-exporter-tls\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.296403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.296381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lln6w\" (UniqueName: \"kubernetes.io/projected/3a860158-6ffe-4345-8c2a-e1661cb29d76-kube-api-access-lln6w\") pod \"node-exporter-dkwzc\" (UID: \"3a860158-6ffe-4345-8c2a-e1661cb29d76\") " pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.325444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.325350 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-dkwzc" Jul 14 17:06:26.398458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:26.398420 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:27.090258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:27.090217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:27.174290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:27.174252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:27.421460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:27.421371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:27.503138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:27.503100 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:28.497266 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:28.496834 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:28.497266 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:28.496917 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:28.502862 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:28.502830 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:28.969398 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:28.969331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:29.069368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:29.069329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:29.150852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:29.150826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:29.172764 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:29.172729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:29.677520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:29.677480 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.058645 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.058551 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8476656947-58b5p" Jul 14 17:06:30.137625 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.137571 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.238208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.238176 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.307675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.307640 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.460267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.460236 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:30.460452 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.460311 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:30.465849 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.465826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:30.603369 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.603320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.686725 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.686688 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.751421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.751331 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:30.825310 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:30.825266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:31.159145 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:31.159112 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:06:31.226963 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:31.226928 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:31.794035 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:31.794000 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a860158_6ffe_4345_8c2a_e1661cb29d76.slice/crio-bc3eef4d4bb43ed5bfaa7d8a8342c4836329bd5a2dad206ee2c5f8b0bb58bc42 WatchSource:0}: Error finding container bc3eef4d4bb43ed5bfaa7d8a8342c4836329bd5a2dad206ee2c5f8b0bb58bc42: Status 404 returned error can't find the container with id bc3eef4d4bb43ed5bfaa7d8a8342c4836329bd5a2dad206ee2c5f8b0bb58bc42 Jul 14 17:06:32.159107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:32.159074 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkwzc" event={"ID":"3a860158-6ffe-4345-8c2a-e1661cb29d76","Type":"ContainerStarted","Data":"bc3eef4d4bb43ed5bfaa7d8a8342c4836329bd5a2dad206ee2c5f8b0bb58bc42"} Jul 14 17:06:32.273460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:32.273428 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:32.413686 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:32.413589 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" podUID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" containerName="registry" containerID="cri-o://ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263" gracePeriod=30 Jul 14 17:06:32.928554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:32.928516 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:06:33.044733 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044636 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx6ld\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044733 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044705 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044733 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044733 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044769 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044818 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044861 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044910 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.044988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.044945 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration\") pod \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\" (UID: \"2592ee91-ae20-4852-ad0a-03b6c3a177eb\") " Jul 14 17:06:33.048129 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.045520 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:33.048129 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.045793 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:33.051031 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.050944 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld" (OuterVolumeSpecName: "kube-api-access-nx6ld") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "kube-api-access-nx6ld". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:33.051174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.051054 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:33.051472 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.051446 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:33.051620 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.051593 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:33.051887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.051852 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:33.057942 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.057833 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2592ee91-ae20-4852-ad0a-03b6c3a177eb" (UID: "2592ee91-ae20-4852-ad0a-03b6c3a177eb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:06:33.146018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.145977 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-certificates\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146019 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-registry-tls\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146033 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-image-registry-private-configuration\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146041 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx6ld\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-kube-api-access-nx6ld\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146054 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2592ee91-ae20-4852-ad0a-03b6c3a177eb-trusted-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146066 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2592ee91-ae20-4852-ad0a-03b6c3a177eb-bound-sa-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146079 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2592ee91-ae20-4852-ad0a-03b6c3a177eb-ca-trust-extracted\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.146258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.146090 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2592ee91-ae20-4852-ad0a-03b6c3a177eb-installation-pull-secrets\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:33.165543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.165507 2574 generic.go:356] "Generic (PLEG): container finished" podID="3a860158-6ffe-4345-8c2a-e1661cb29d76" containerID="516d7c8102fafc8cf7b734dc8d889bda68498bddab0ab49b79f55c9a72437128" exitCode=0 Jul 14 17:06:33.165746 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.165616 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkwzc" event={"ID":"3a860158-6ffe-4345-8c2a-e1661cb29d76","Type":"ContainerDied","Data":"516d7c8102fafc8cf7b734dc8d889bda68498bddab0ab49b79f55c9a72437128"} Jul 14 17:06:33.167550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.167525 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lp9p6" event={"ID":"742b8fa1-9d7e-4e17-8fb1-5f211689026d","Type":"ContainerStarted","Data":"604f2806a4dbc0d34623c4e9459264552796ef9545b5c4834a6f2a5e7890a9e3"} Jul 14 17:06:33.167722 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.167704 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:33.169780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.169752 2574 generic.go:356] "Generic (PLEG): container finished" podID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" containerID="ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263" exitCode=0 Jul 14 17:06:33.169887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.169798 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" Jul 14 17:06:33.169887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.169807 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" event={"ID":"2592ee91-ae20-4852-ad0a-03b6c3a177eb","Type":"ContainerDied","Data":"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263"} Jul 14 17:06:33.169887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.169857 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr" event={"ID":"2592ee91-ae20-4852-ad0a-03b6c3a177eb","Type":"ContainerDied","Data":"18cb62f11fe936ca859e1d4df849508019902fdfcee5feafb17e0629128762cd"} Jul 14 17:06:33.170054 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.169895 2574 scope.go:117] "RemoveContainer" containerID="ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263" Jul 14 17:06:33.174285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.172283 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-lp9p6" Jul 14 17:06:33.186441 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.186417 2574 scope.go:117] "RemoveContainer" containerID="ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263" Jul 14 17:06:33.186845 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:06:33.186821 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263\": container with ID starting with ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263 not found: ID does not exist" containerID="ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263" Jul 14 17:06:33.186942 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.186858 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263"} err="failed to get container status \"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263\": rpc error: code = NotFound desc = could not find container \"ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263\": container with ID starting with ad636df7933caaa1ac17aeaa149be54edb52ee5c63631c2109125329c5f48263 not found: ID does not exist" Jul 14 17:06:33.206461 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.205739 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-lp9p6" podStartSLOduration=2.560973643 podStartE2EDuration="25.205715079s" totalImagesPullingTime="22.644741436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:33.205609018 +0000 UTC m=+86.728549791" watchObservedRunningTime="2026-07-14 17:06:33.205715079 +0000 UTC m=+86.728655854" Jul 14 17:06:33.222311 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.221954 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:06:33.227592 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.227540 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cfb8c4c5d-f9nhr"] Jul 14 17:06:33.556185 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.556145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:33.629109 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.629071 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:33.693221 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:33.693184 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" path="/var/lib/kubelet/pods/2592ee91-ae20-4852-ad0a-03b6c3a177eb/volumes" Jul 14 17:06:34.180128 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.180091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkwzc" event={"ID":"3a860158-6ffe-4345-8c2a-e1661cb29d76","Type":"ContainerStarted","Data":"9378813897ab7af0f077cebd1c26dee994ae2fd2042debbb3b4417657ef1c20c"} Jul 14 17:06:34.180673 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.180147 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-dkwzc" event={"ID":"3a860158-6ffe-4345-8c2a-e1661cb29d76","Type":"ContainerStarted","Data":"7a1998cb62f9370796c48d1ffb6a8cce1ac0688377da4132b604f9dd03abd776"} Jul 14 17:06:34.205390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.205314 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-dkwzc" podStartSLOduration=8.074932805 podStartE2EDuration="9.205290943s" totalImagesPullingTime="1.011164198s" totalInitContainerRuntime="119.19394ms" isStatefulPod=true podCreationTimestamp="2026-07-14 17:06:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:34.204831201 +0000 UTC m=+87.727771975" watchObservedRunningTime="2026-07-14 17:06:34.205290943 +0000 UTC m=+87.728231716" Jul 14 17:06:34.551901 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.551809 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:06:34.552127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.552112 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" containerName="registry" Jul 14 17:06:34.552182 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.552130 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" containerName="registry" Jul 14 17:06:34.552182 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.552181 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2592ee91-ae20-4852-ad0a-03b6c3a177eb" containerName="registry" Jul 14 17:06:34.577403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.577373 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:06:34.577568 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.577489 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659642 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659860 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659763 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659860 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659805 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659860 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659991 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659866 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.659991 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h69wx\" (UniqueName: \"kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.660078 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.659998 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.756502 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.756462 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:34.760404 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760419 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h69wx\" (UniqueName: \"kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760453 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760519 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760554 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760791 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.760791 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.760637 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.761552 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.761527 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.761746 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.761691 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.762298 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.762269 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.762384 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.762269 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.763530 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.763501 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.763697 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.763676 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.771245 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.771216 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h69wx\" (UniqueName: \"kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx\") pod \"console-6f64b4d7c7-jls6k\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:34.888875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:34.888836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:35.064934 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:35.064889 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:06:35.068667 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:35.068631 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861d154d_527f_4e22_b9ee_80dff4af3518.slice/crio-e846dd2e7fa1433d830b2ba6c93a73522b3b2b20140782181043cbb92972305c WatchSource:0}: Error finding container e846dd2e7fa1433d830b2ba6c93a73522b3b2b20140782181043cbb92972305c: Status 404 returned error can't find the container with id e846dd2e7fa1433d830b2ba6c93a73522b3b2b20140782181043cbb92972305c Jul 14 17:06:35.185619 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:35.185559 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f64b4d7c7-jls6k" event={"ID":"861d154d-527f-4e22-b9ee-80dff4af3518","Type":"ContainerStarted","Data":"e846dd2e7fa1433d830b2ba6c93a73522b3b2b20140782181043cbb92972305c"} Jul 14 17:06:36.192012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:36.191970 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f64b4d7c7-jls6k" event={"ID":"861d154d-527f-4e22-b9ee-80dff4af3518","Type":"ContainerStarted","Data":"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a"} Jul 14 17:06:36.213372 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:36.213308 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f64b4d7c7-jls6k" podStartSLOduration=2.213289193 podStartE2EDuration="2.213289193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:36.21176778 +0000 UTC m=+89.734708556" watchObservedRunningTime="2026-07-14 17:06:36.213289193 +0000 UTC m=+89.736229965" Jul 14 17:06:36.405228 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:36.405188 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:36.501623 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:36.501514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:38.042352 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.042316 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:06:38.077460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.077422 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:06:38.095992 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.095954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:38.114305 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.114271 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:06:38.114494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.114383 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.198390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.198346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:38.295029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.294938 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9fs\" (UniqueName: \"kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295242 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295242 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295158 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295242 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295222 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.295390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.295303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.300487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.300455 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:38.395780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395743 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395835 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9fs\" (UniqueName: \"kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395886 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395920 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.395982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.395968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.396785 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.396760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.397367 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.397335 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.397367 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.397358 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.397700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.397679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.398858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.398837 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.399008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.398984 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.405642 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.405614 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9fs\" (UniqueName: \"kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs\") pod \"console-859984dbf5-h69fc\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.430426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.430389 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:38.615694 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.615657 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:06:38.618419 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:06:38.618387 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f4d089_7823_466f_999f_b90a879deb47.slice/crio-b2d67895c908259beaf6f3e98f59d54061edca437042db06c1d1b98e66e1f7d8 WatchSource:0}: Error finding container b2d67895c908259beaf6f3e98f59d54061edca437042db06c1d1b98e66e1f7d8: Status 404 returned error can't find the container with id b2d67895c908259beaf6f3e98f59d54061edca437042db06c1d1b98e66e1f7d8 Jul 14 17:06:38.631377 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.631330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:38.715118 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:38.715087 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:39.205740 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:39.205695 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859984dbf5-h69fc" event={"ID":"37f4d089-7823-466f-999f-b90a879deb47","Type":"ContainerStarted","Data":"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf"} Jul 14 17:06:39.206188 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:39.205763 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859984dbf5-h69fc" event={"ID":"37f4d089-7823-466f-999f-b90a879deb47","Type":"ContainerStarted","Data":"b2d67895c908259beaf6f3e98f59d54061edca437042db06c1d1b98e66e1f7d8"} Jul 14 17:06:39.227920 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:39.227856 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-859984dbf5-h69fc" podStartSLOduration=1.227832987 podStartE2EDuration="1.227832987s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:06:39.226761583 +0000 UTC m=+92.749702360" watchObservedRunningTime="2026-07-14 17:06:39.227832987 +0000 UTC m=+92.750773758" Jul 14 17:06:39.234439 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:39.234399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:39.294908 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:39.294873 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:41.399536 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:41.399493 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:41.501645 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:41.501612 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:44.889890 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:44.889854 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:06:48.431080 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:48.431040 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:48.431530 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:48.431106 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:48.435863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:48.435838 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:49.252267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:49.252239 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:06:49.307854 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:49.307816 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:06:49.426207 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:49.426160 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:49.527453 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:49.527346 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:49.637415 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:49.637380 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:06:51.114675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:51.114647 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nd78m" Jul 14 17:06:56.251713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.251652 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68679bb776-zqd8m" podUID="9e06efe5-093a-48de-a48b-97ec7e9d1979" containerName="console" containerID="cri-o://e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1" gracePeriod=15 Jul 14 17:06:56.529168 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.529134 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:56.552713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552685 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.552885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552725 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4kdj\" (UniqueName: \"kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.552885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552749 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.552885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552782 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.553047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552918 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.553047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.552982 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca\") pod \"9e06efe5-093a-48de-a48b-97ec7e9d1979\" (UID: \"9e06efe5-093a-48de-a48b-97ec7e9d1979\") " Jul 14 17:06:56.553192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.553167 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:56.553192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.553179 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config" (OuterVolumeSpecName: "console-config") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:56.553334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.553250 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.553334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.553268 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-oauth-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.553461 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.553444 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca" (OuterVolumeSpecName: "service-ca") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:06:56.555370 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.555339 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:56.555518 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.555401 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj" (OuterVolumeSpecName: "kube-api-access-q4kdj") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "kube-api-access-q4kdj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:06:56.555621 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.555596 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9e06efe5-093a-48de-a48b-97ec7e9d1979" (UID: "9e06efe5-093a-48de-a48b-97ec7e9d1979"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:06:56.654278 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.654236 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.654278 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.654272 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9e06efe5-093a-48de-a48b-97ec7e9d1979-console-oauth-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.654278 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.654283 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9e06efe5-093a-48de-a48b-97ec7e9d1979-service-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:56.654510 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:56.654292 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4kdj\" (UniqueName: \"kubernetes.io/projected/9e06efe5-093a-48de-a48b-97ec7e9d1979-kube-api-access-q4kdj\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:06:57.278891 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.278857 2574 generic.go:356] "Generic (PLEG): container finished" podID="9e06efe5-093a-48de-a48b-97ec7e9d1979" containerID="e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1" exitCode=0 Jul 14 17:06:57.279334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.278900 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68679bb776-zqd8m" Jul 14 17:06:57.279334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.278910 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68679bb776-zqd8m" event={"ID":"9e06efe5-093a-48de-a48b-97ec7e9d1979","Type":"ContainerDied","Data":"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1"} Jul 14 17:06:57.279334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.278951 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68679bb776-zqd8m" event={"ID":"9e06efe5-093a-48de-a48b-97ec7e9d1979","Type":"ContainerDied","Data":"c36921fb601955e2c980f71b043a33936a40f958e58158a59791b0b77b960383"} Jul 14 17:06:57.279334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.278971 2574 scope.go:117] "RemoveContainer" containerID="e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1" Jul 14 17:06:57.294749 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.294720 2574 scope.go:117] "RemoveContainer" containerID="e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1" Jul 14 17:06:57.295053 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:06:57.295031 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1\": container with ID starting with e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1 not found: ID does not exist" containerID="e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1" Jul 14 17:06:57.295114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.295063 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1"} err="failed to get container status \"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1\": rpc error: code = NotFound desc = could not find container \"e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1\": container with ID starting with e95704e0fea81d91037bd5e964babc4cbab8fefc3a985a09bf5c52be605427a1 not found: ID does not exist" Jul 14 17:06:57.310668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.310639 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:57.313691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.313668 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68679bb776-zqd8m"] Jul 14 17:06:57.691311 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:06:57.691277 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e06efe5-093a-48de-a48b-97ec7e9d1979" path="/var/lib/kubelet/pods/9e06efe5-093a-48de-a48b-97ec7e9d1979/volumes" Jul 14 17:07:03.222965 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.222899 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f64b4d7c7-jls6k" podUID="861d154d-527f-4e22-b9ee-80dff4af3518" containerName="console" containerID="cri-o://0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a" gracePeriod=15 Jul 14 17:07:03.504535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.504494 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:07:03.607208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607167 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607216 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h69wx\" (UniqueName: \"kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607438 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607258 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607438 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607290 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607432 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607486 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607528 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert\") pod \"861d154d-527f-4e22-b9ee-80dff4af3518\" (UID: \"861d154d-527f-4e22-b9ee-80dff4af3518\") " Jul 14 17:07:03.607710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607653 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config" (OuterVolumeSpecName: "console-config") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:03.607793 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607762 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca" (OuterVolumeSpecName: "service-ca") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:03.607962 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607821 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-console-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.607962 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.607890 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:03.608046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.608003 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:03.609505 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.609482 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:03.609621 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.609524 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx" (OuterVolumeSpecName: "kube-api-access-h69wx") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "kube-api-access-h69wx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:03.609621 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.609569 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "861d154d-527f-4e22-b9ee-80dff4af3518" (UID: "861d154d-527f-4e22-b9ee-80dff4af3518"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:03.708457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708416 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.708457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708450 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-oauth-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.708457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708459 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h69wx\" (UniqueName: \"kubernetes.io/projected/861d154d-527f-4e22-b9ee-80dff4af3518-kube-api-access-h69wx\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.708696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708468 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-service-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.708696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708477 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/861d154d-527f-4e22-b9ee-80dff4af3518-trusted-ca-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.708696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.708486 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/861d154d-527f-4e22-b9ee-80dff4af3518-console-oauth-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:03.898933 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:03.898896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:04.000788 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.000744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:04.307484 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.307388 2574 generic.go:356] "Generic (PLEG): container finished" podID="861d154d-527f-4e22-b9ee-80dff4af3518" containerID="0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a" exitCode=0 Jul 14 17:07:04.307484 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.307444 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f64b4d7c7-jls6k" Jul 14 17:07:04.307484 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.307475 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f64b4d7c7-jls6k" event={"ID":"861d154d-527f-4e22-b9ee-80dff4af3518","Type":"ContainerDied","Data":"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a"} Jul 14 17:07:04.308000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.307548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f64b4d7c7-jls6k" event={"ID":"861d154d-527f-4e22-b9ee-80dff4af3518","Type":"ContainerDied","Data":"e846dd2e7fa1433d830b2ba6c93a73522b3b2b20140782181043cbb92972305c"} Jul 14 17:07:04.308000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.307570 2574 scope.go:117] "RemoveContainer" containerID="0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a" Jul 14 17:07:04.315344 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.315327 2574 scope.go:117] "RemoveContainer" containerID="0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a" Jul 14 17:07:04.315626 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:07:04.315606 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a\": container with ID starting with 0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a not found: ID does not exist" containerID="0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a" Jul 14 17:07:04.315690 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.315638 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a"} err="failed to get container status \"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a\": rpc error: code = NotFound desc = could not find container \"0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a\": container with ID starting with 0a02aace1c23972586ce613973210ee9d35ae79a38859200430709eb8367d07a not found: ID does not exist" Jul 14 17:07:04.331199 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.331168 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:07:04.343590 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:04.343549 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f64b4d7c7-jls6k"] Jul 14 17:07:05.691298 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:05.691262 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="861d154d-527f-4e22-b9ee-80dff4af3518" path="/var/lib/kubelet/pods/861d154d-527f-4e22-b9ee-80dff4af3518/volumes" Jul 14 17:07:14.327788 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.327747 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ddcdf4c6b-svr4x" podUID="1ae4e9d8-8cef-444a-9034-87771d159372" containerName="console" containerID="cri-o://9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f" gracePeriod=15 Jul 14 17:07:14.604703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.604635 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:07:14.705093 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705056 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2lbl\" (UniqueName: \"kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705093 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705091 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705119 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705136 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705164 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705198 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705286 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle\") pod \"1ae4e9d8-8cef-444a-9034-87771d159372\" (UID: \"1ae4e9d8-8cef-444a-9034-87771d159372\") " Jul 14 17:07:14.705618 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705565 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:14.705618 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705594 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config" (OuterVolumeSpecName: "console-config") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:14.705728 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705613 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca" (OuterVolumeSpecName: "service-ca") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:14.705799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.705782 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:07:14.707321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.707298 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:14.707738 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.707715 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:07:14.707794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.707774 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl" (OuterVolumeSpecName: "kube-api-access-q2lbl") pod "1ae4e9d8-8cef-444a-9034-87771d159372" (UID: "1ae4e9d8-8cef-444a-9034-87771d159372"). InnerVolumeSpecName "kube-api-access-q2lbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:07:14.806499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806457 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-service-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806495 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806507 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ae4e9d8-8cef-444a-9034-87771d159372-console-oauth-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806521 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-trusted-ca-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806534 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2lbl\" (UniqueName: \"kubernetes.io/projected/1ae4e9d8-8cef-444a-9034-87771d159372-kube-api-access-q2lbl\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806548 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-console-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:14.806819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:14.806562 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ae4e9d8-8cef-444a-9034-87771d159372-oauth-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:07:15.272626 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.272592 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:15.349471 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.349437 2574 generic.go:356] "Generic (PLEG): container finished" podID="1ae4e9d8-8cef-444a-9034-87771d159372" containerID="9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f" exitCode=0 Jul 14 17:07:15.349471 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.349474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcdf4c6b-svr4x" event={"ID":"1ae4e9d8-8cef-444a-9034-87771d159372","Type":"ContainerDied","Data":"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f"} Jul 14 17:07:15.349998 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.349484 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcdf4c6b-svr4x" Jul 14 17:07:15.349998 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.349507 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcdf4c6b-svr4x" event={"ID":"1ae4e9d8-8cef-444a-9034-87771d159372","Type":"ContainerDied","Data":"4c5bed6058ade8b35a94591d4c8cd5f06b8018ecb71f4f666df3687589fa57e1"} Jul 14 17:07:15.349998 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.349526 2574 scope.go:117] "RemoveContainer" containerID="9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f" Jul 14 17:07:15.358099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.358081 2574 scope.go:117] "RemoveContainer" containerID="9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f" Jul 14 17:07:15.358404 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:07:15.358382 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f\": container with ID starting with 9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f not found: ID does not exist" containerID="9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f" Jul 14 17:07:15.358454 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.358416 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f"} err="failed to get container status \"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f\": rpc error: code = NotFound desc = could not find container \"9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f\": container with ID starting with 9f5000479d95682d9c251d5db8d0297da569aab924babd172d798f2be5a0d58f not found: ID does not exist" Jul 14 17:07:15.374518 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.374490 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:07:15.383633 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.381736 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ddcdf4c6b-svr4x"] Jul 14 17:07:15.691332 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:15.691291 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ae4e9d8-8cef-444a-9034-87771d159372" path="/var/lib/kubelet/pods/1ae4e9d8-8cef-444a-9034-87771d159372/volumes" Jul 14 17:07:29.149816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:29.149772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:42.443964 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:42.443926 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:42.512234 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:42.512191 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:53.294839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.294800 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295176 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ae4e9d8-8cef-444a-9034-87771d159372" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295196 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ae4e9d8-8cef-444a-9034-87771d159372" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295214 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="861d154d-527f-4e22-b9ee-80dff4af3518" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295222 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="861d154d-527f-4e22-b9ee-80dff4af3518" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295237 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e06efe5-093a-48de-a48b-97ec7e9d1979" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295245 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e06efe5-093a-48de-a48b-97ec7e9d1979" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295312 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="861d154d-527f-4e22-b9ee-80dff4af3518" containerName="console" Jul 14 17:07:53.295321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295323 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ae4e9d8-8cef-444a-9034-87771d159372" containerName="console" Jul 14 17:07:53.295725 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.295333 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e06efe5-093a-48de-a48b-97ec7e9d1979" containerName="console" Jul 14 17:07:53.297375 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.297355 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.313202 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.313168 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:07:53.354484 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.354446 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:53.407297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407434 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407434 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407330 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkc8z\" (UniqueName: \"kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407434 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407380 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407434 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.407622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.407472 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.417433 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.417402 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:53.507957 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.507914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.507957 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.507960 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.507981 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508008 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508049 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkc8z\" (UniqueName: \"kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508824 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508785 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508926 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508879 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.508967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.508956 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.509177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.509154 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.510503 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.510485 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.510671 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.510648 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.517708 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.517681 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkc8z\" (UniqueName: \"kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z\") pod \"console-797c458578-f4trg\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.607586 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.607551 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:07:53.740049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:53.740014 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:07:53.744482 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:07:53.744453 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2c9056_5205_457c_86cf_cccd6a8891e1.slice/crio-cd5b477d969bac07a2ae04e124555eae7332a611a338582b8072b9f7b3ed8b02 WatchSource:0}: Error finding container cd5b477d969bac07a2ae04e124555eae7332a611a338582b8072b9f7b3ed8b02: Status 404 returned error can't find the container with id cd5b477d969bac07a2ae04e124555eae7332a611a338582b8072b9f7b3ed8b02 Jul 14 17:07:54.191111 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:54.191082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:54.292050 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:54.292016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:07:54.484722 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:54.484631 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c458578-f4trg" event={"ID":"bb2c9056-5205-457c-86cf-cccd6a8891e1","Type":"ContainerStarted","Data":"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2"} Jul 14 17:07:54.484722 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:54.484686 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c458578-f4trg" event={"ID":"bb2c9056-5205-457c-86cf-cccd6a8891e1","Type":"ContainerStarted","Data":"cd5b477d969bac07a2ae04e124555eae7332a611a338582b8072b9f7b3ed8b02"} Jul 14 17:07:54.505064 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:07:54.505010 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797c458578-f4trg" podStartSLOduration=1.5049918039999999 podStartE2EDuration="1.504991804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:07:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:07:54.503868376 +0000 UTC m=+168.026809150" watchObservedRunningTime="2026-07-14 17:07:54.504991804 +0000 UTC m=+168.027932624" Jul 14 17:08:03.608142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:03.608049 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:08:03.608142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:03.608117 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:08:03.613007 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:03.612975 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:08:04.523745 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:04.523712 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:08:04.589324 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:04.589295 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:08:04.708700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:04.708664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:04.813086 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:04.813000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:04.912134 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:04.912094 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:11.577372 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:11.577335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:11.678330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:11.678296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:26.080411 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:26.080370 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:26.179715 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:26.179675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:08:29.609255 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.609212 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-859984dbf5-h69fc" podUID="37f4d089-7823-466f-999f-b90a879deb47" containerName="console" containerID="cri-o://e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf" gracePeriod=15 Jul 14 17:08:29.846471 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.846447 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:08:29.987407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987291 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987351 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987382 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987465 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987493 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987539 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987562 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p9fs\" (UniqueName: \"kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs\") pod \"37f4d089-7823-466f-999f-b90a879deb47\" (UID: \"37f4d089-7823-466f-999f-b90a879deb47\") " Jul 14 17:08:29.987913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987759 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca" (OuterVolumeSpecName: "service-ca") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:08:29.987913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987878 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:08:29.987913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987898 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-service-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:29.987913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.987895 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:08:29.988120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.988092 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config" (OuterVolumeSpecName: "console-config") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:08:29.989765 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.989738 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:08:29.989882 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.989777 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:08:29.989882 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:29.989857 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs" (OuterVolumeSpecName: "kube-api-access-6p9fs") pod "37f4d089-7823-466f-999f-b90a879deb47" (UID: "37f4d089-7823-466f-999f-b90a879deb47"). InnerVolumeSpecName "kube-api-access-6p9fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:08:30.088589 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088548 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-oauth-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.088750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088604 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-oauth-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.088750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088614 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37f4d089-7823-466f-999f-b90a879deb47-console-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.088750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088624 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-console-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.088750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088634 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p9fs\" (UniqueName: \"kubernetes.io/projected/37f4d089-7823-466f-999f-b90a879deb47-kube-api-access-6p9fs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.088750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.088644 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37f4d089-7823-466f-999f-b90a879deb47-trusted-ca-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:08:30.612795 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.612763 2574 generic.go:356] "Generic (PLEG): container finished" podID="37f4d089-7823-466f-999f-b90a879deb47" containerID="e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf" exitCode=0 Jul 14 17:08:30.613159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.612813 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859984dbf5-h69fc" Jul 14 17:08:30.613159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.612815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859984dbf5-h69fc" event={"ID":"37f4d089-7823-466f-999f-b90a879deb47","Type":"ContainerDied","Data":"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf"} Jul 14 17:08:30.613159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.612860 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859984dbf5-h69fc" event={"ID":"37f4d089-7823-466f-999f-b90a879deb47","Type":"ContainerDied","Data":"b2d67895c908259beaf6f3e98f59d54061edca437042db06c1d1b98e66e1f7d8"} Jul 14 17:08:30.613159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.612880 2574 scope.go:117] "RemoveContainer" containerID="e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf" Jul 14 17:08:30.621101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.621084 2574 scope.go:117] "RemoveContainer" containerID="e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf" Jul 14 17:08:30.621377 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:08:30.621357 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf\": container with ID starting with e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf not found: ID does not exist" containerID="e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf" Jul 14 17:08:30.621424 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.621387 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf"} err="failed to get container status \"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf\": rpc error: code = NotFound desc = could not find container \"e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf\": container with ID starting with e7f60b997c71ba4b331e406828c89bdb2b394d6773146dd02d5080ab278bd0bf not found: ID does not exist" Jul 14 17:08:30.634159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.634129 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:08:30.636423 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:30.636402 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-859984dbf5-h69fc"] Jul 14 17:08:31.691077 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:08:31.691038 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f4d089-7823-466f-999f-b90a879deb47" path="/var/lib/kubelet/pods/37f4d089-7823-466f-999f-b90a879deb47/volumes" Jul 14 17:09:01.660865 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:01.660829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:01.759569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:01.759504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:01.860172 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:01.860134 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:02.366324 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.366292 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq"] Jul 14 17:09:02.366561 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.366550 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f4d089-7823-466f-999f-b90a879deb47" containerName="console" Jul 14 17:09:02.366626 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.366563 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f4d089-7823-466f-999f-b90a879deb47" containerName="console" Jul 14 17:09:02.366667 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.366637 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f4d089-7823-466f-999f-b90a879deb47" containerName="console" Jul 14 17:09:02.368602 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.368565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.371240 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.371214 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:09:02.371376 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.371287 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:09:02.372389 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.372365 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9qc8\"" Jul 14 17:09:02.380057 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.380031 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq"] Jul 14 17:09:02.412264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.412216 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.412264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.412265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgwvd\" (UniqueName: \"kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.412497 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.412304 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.513107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.513067 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.513107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.513109 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgwvd\" (UniqueName: \"kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.513356 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.513143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.513481 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.513463 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.513549 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.513510 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.523381 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.523350 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgwvd\" (UniqueName: \"kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.677738 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.677636 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:02.809640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:02.809604 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq"] Jul 14 17:09:02.813377 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:02.813348 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d722a6b_9c45_4a2c_bca3_2985ed72724b.slice/crio-706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb WatchSource:0}: Error finding container 706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb: Status 404 returned error can't find the container with id 706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb Jul 14 17:09:03.727636 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:03.727600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" event={"ID":"0d722a6b-9c45-4a2c-bca3-2985ed72724b","Type":"ContainerStarted","Data":"706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb"} Jul 14 17:09:11.506100 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:11.506057 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:11.567919 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:11.567885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:11.756566 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:11.756478 2574 generic.go:356] "Generic (PLEG): container finished" podID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerID="8148c0c9bc3dd4a01eae070957697c4ae0acfd815d70d7d27c87b39920af58e3" exitCode=0 Jul 14 17:09:11.756729 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:11.756564 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" event={"ID":"0d722a6b-9c45-4a2c-bca3-2985ed72724b","Type":"ContainerDied","Data":"8148c0c9bc3dd4a01eae070957697c4ae0acfd815d70d7d27c87b39920af58e3"} Jul 14 17:09:14.769127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:14.769086 2574 generic.go:356] "Generic (PLEG): container finished" podID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerID="98de4d24aba95b12bad10d3fea7fa1471c85e63f9f36acfe24fb10974af80028" exitCode=0 Jul 14 17:09:14.769498 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:14.769177 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" event={"ID":"0d722a6b-9c45-4a2c-bca3-2985ed72724b","Type":"ContainerDied","Data":"98de4d24aba95b12bad10d3fea7fa1471c85e63f9f36acfe24fb10974af80028"} Jul 14 17:09:23.806068 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:23.806033 2574 generic.go:356] "Generic (PLEG): container finished" podID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerID="dcdb2131850ceebc122d5d314bcd9c73610ba0c7e25be620d93828159e9f3280" exitCode=0 Jul 14 17:09:23.806448 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:23.806111 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" event={"ID":"0d722a6b-9c45-4a2c-bca3-2985ed72724b","Type":"ContainerDied","Data":"dcdb2131850ceebc122d5d314bcd9c73610ba0c7e25be620d93828159e9f3280"} Jul 14 17:09:24.932364 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.932339 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:24.987878 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.987842 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util\") pod \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " Jul 14 17:09:24.988047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.987907 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgwvd\" (UniqueName: \"kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd\") pod \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " Jul 14 17:09:24.988047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.987932 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle\") pod \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\" (UID: \"0d722a6b-9c45-4a2c-bca3-2985ed72724b\") " Jul 14 17:09:24.988627 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.988603 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle" (OuterVolumeSpecName: "bundle") pod "0d722a6b-9c45-4a2c-bca3-2985ed72724b" (UID: "0d722a6b-9c45-4a2c-bca3-2985ed72724b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:09:24.990180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.990145 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd" (OuterVolumeSpecName: "kube-api-access-bgwvd") pod "0d722a6b-9c45-4a2c-bca3-2985ed72724b" (UID: "0d722a6b-9c45-4a2c-bca3-2985ed72724b"). InnerVolumeSpecName "kube-api-access-bgwvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:09:24.992649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:24.992617 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util" (OuterVolumeSpecName: "util") pod "0d722a6b-9c45-4a2c-bca3-2985ed72724b" (UID: "0d722a6b-9c45-4a2c-bca3-2985ed72724b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:09:25.088448 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.088417 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:25.088448 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.088447 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgwvd\" (UniqueName: \"kubernetes.io/projected/0d722a6b-9c45-4a2c-bca3-2985ed72724b-kube-api-access-bgwvd\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:25.088668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.088459 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d722a6b-9c45-4a2c-bca3-2985ed72724b-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:25.813821 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.813786 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" event={"ID":"0d722a6b-9c45-4a2c-bca3-2985ed72724b","Type":"ContainerDied","Data":"706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb"} Jul 14 17:09:25.813987 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.813832 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="706096f633a9b774a57984bbb80dafb44241292c328048ef5a20d22328ed11fb" Jul 14 17:09:25.813987 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:25.813830 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540nj4hq" Jul 14 17:09:28.507486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:28.507401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:29.449662 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:29.449614 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:30.386928 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.386899 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq"] Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387145 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="extract" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387155 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="extract" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387171 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="util" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387176 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="util" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387183 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="pull" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387189 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="pull" Jul 14 17:09:30.387289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.387234 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d722a6b-9c45-4a2c-bca3-2985ed72724b" containerName="extract" Jul 14 17:09:30.389137 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.389121 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.391846 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.391825 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 17:09:30.391975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.391829 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:09:30.391975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.391878 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-fqchm\"" Jul 14 17:09:30.405707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.405676 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq"] Jul 14 17:09:30.531630 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.531533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.531807 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.531674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9sd\" (UniqueName: \"kubernetes.io/projected/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-kube-api-access-vk9sd\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.632946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.632902 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9sd\" (UniqueName: \"kubernetes.io/projected/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-kube-api-access-vk9sd\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.633117 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.632966 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.633276 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.633261 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.642066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.642010 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9sd\" (UniqueName: \"kubernetes.io/projected/2b40c9a8-8efe-4691-a98a-4199bf7b6a72-kube-api-access-vk9sd\") pod \"cert-manager-operator-controller-manager-55c444bf86-8c2vq\" (UID: \"2b40c9a8-8efe-4691-a98a-4199bf7b6a72\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.698342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.698307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" Jul 14 17:09:30.846622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:30.846465 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq"] Jul 14 17:09:30.849388 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:30.849359 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b40c9a8_8efe_4691_a98a_4199bf7b6a72.slice/crio-5c14ab996bf48ec3bfb2fb2e640556e826e971219a4ff453d21eb7076f121771 WatchSource:0}: Error finding container 5c14ab996bf48ec3bfb2fb2e640556e826e971219a4ff453d21eb7076f121771: Status 404 returned error can't find the container with id 5c14ab996bf48ec3bfb2fb2e640556e826e971219a4ff453d21eb7076f121771 Jul 14 17:09:31.835945 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:31.835914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" event={"ID":"2b40c9a8-8efe-4691-a98a-4199bf7b6a72","Type":"ContainerStarted","Data":"5c14ab996bf48ec3bfb2fb2e640556e826e971219a4ff453d21eb7076f121771"} Jul 14 17:09:32.085275 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:32.085235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:32.269571 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:32.269461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:32.841526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:32.841490 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" event={"ID":"2b40c9a8-8efe-4691-a98a-4199bf7b6a72","Type":"ContainerStarted","Data":"ff71751a382cc8750e4e749d76d64b467fa6f4a8713703ad2664a48d13309382"} Jul 14 17:09:32.880330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:32.880267 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8c2vq" podStartSLOduration=1.230234531 podStartE2EDuration="2.88024366s" totalImagesPullingTime="1.650009129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:09:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:09:32.878989733 +0000 UTC m=+266.401930506" watchObservedRunningTime="2026-07-14 17:09:32.88024366 +0000 UTC m=+266.403184417" Jul 14 17:09:33.359859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:33.359818 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:33.507755 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:33.507714 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:34.361078 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.361041 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl"] Jul 14 17:09:34.363856 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.363835 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.366754 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.366733 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9qc8\"" Jul 14 17:09:34.367908 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.367892 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:09:34.367978 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.367926 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:09:34.380004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.379974 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl"] Jul 14 17:09:34.385011 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.384969 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:34.467554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.467510 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82npv\" (UniqueName: \"kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.467741 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.467649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.467741 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.467674 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.530594 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.530530 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:34.568402 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.568362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.568402 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.568406 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.568629 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.568443 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82npv\" (UniqueName: \"kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.568782 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.568764 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.568821 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.568804 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.597276 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.597238 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82npv\" (UniqueName: \"kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.673198 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.673091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:34.828143 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.828108 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl"] Jul 14 17:09:34.833763 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:34.833730 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bd3c43_afed_4a52_a901_c855842b09e8.slice/crio-fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae WatchSource:0}: Error finding container fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae: Status 404 returned error can't find the container with id fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae Jul 14 17:09:34.849116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:34.849085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" event={"ID":"24bd3c43-afed-4a52-a901-c855842b09e8","Type":"ContainerStarted","Data":"fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae"} Jul 14 17:09:34.937287 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:09:34.937257 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bd3c43_afed_4a52_a901_c855842b09e8.slice/crio-be447de01a9debd265e608f20cc67f0afb3ceed4ecf3c67501a9ed3b6243542c.scope\": RecentStats: unable to find data in memory cache]" Jul 14 17:09:34.937407 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:09:34.937260 2574 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bd3c43_afed_4a52_a901_c855842b09e8.slice/crio-conmon-be447de01a9debd265e608f20cc67f0afb3ceed4ecf3c67501a9ed3b6243542c.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24bd3c43_afed_4a52_a901_c855842b09e8.slice/crio-be447de01a9debd265e608f20cc67f0afb3ceed4ecf3c67501a9ed3b6243542c.scope\": RecentStats: unable to find data in memory cache]" Jul 14 17:09:35.351391 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.351357 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cjdv7"] Jul 14 17:09:35.353490 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.353473 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.356425 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.356401 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 17:09:35.357453 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.357436 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jc947\"" Jul 14 17:09:35.357536 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.357436 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 17:09:35.369061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.369030 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cjdv7"] Jul 14 17:09:35.481650 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.481620 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.481816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.481676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwqvp\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-kube-api-access-mwqvp\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.582818 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.582784 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwqvp\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-kube-api-access-mwqvp\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.582962 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.582836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.595314 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.595279 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.595447 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.595416 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwqvp\" (UniqueName: \"kubernetes.io/projected/ee36e8c5-c5fc-4acf-bf74-2c6123082f8f-kube-api-access-mwqvp\") pod \"cert-manager-webhook-6486ffd997-cjdv7\" (UID: \"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.675028 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.674989 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:35.809939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.809850 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-cjdv7"] Jul 14 17:09:35.812232 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:35.812205 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee36e8c5_c5fc_4acf_bf74_2c6123082f8f.slice/crio-50b8c8a28b2d8ddcedc037efdc75162489902a31b78e1ac7605f654e01e04192 WatchSource:0}: Error finding container 50b8c8a28b2d8ddcedc037efdc75162489902a31b78e1ac7605f654e01e04192: Status 404 returned error can't find the container with id 50b8c8a28b2d8ddcedc037efdc75162489902a31b78e1ac7605f654e01e04192 Jul 14 17:09:35.854916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.854828 2574 generic.go:356] "Generic (PLEG): container finished" podID="24bd3c43-afed-4a52-a901-c855842b09e8" containerID="be447de01a9debd265e608f20cc67f0afb3ceed4ecf3c67501a9ed3b6243542c" exitCode=0 Jul 14 17:09:35.855058 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.854914 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" event={"ID":"24bd3c43-afed-4a52-a901-c855842b09e8","Type":"ContainerDied","Data":"be447de01a9debd265e608f20cc67f0afb3ceed4ecf3c67501a9ed3b6243542c"} Jul 14 17:09:35.856170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:35.856146 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" event={"ID":"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f","Type":"ContainerStarted","Data":"50b8c8a28b2d8ddcedc037efdc75162489902a31b78e1ac7605f654e01e04192"} Jul 14 17:09:36.784238 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:36.784200 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:36.965126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:36.965088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:38.862735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:38.862681 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:38.870833 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:38.870799 2574 generic.go:356] "Generic (PLEG): container finished" podID="24bd3c43-afed-4a52-a901-c855842b09e8" containerID="291e311c8e5143d8e96a2ba42aedeee0102a79a3630e8fa4663cab04a0131a1c" exitCode=0 Jul 14 17:09:38.871013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:38.870906 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" event={"ID":"24bd3c43-afed-4a52-a901-c855842b09e8","Type":"ContainerDied","Data":"291e311c8e5143d8e96a2ba42aedeee0102a79a3630e8fa4663cab04a0131a1c"} Jul 14 17:09:38.948077 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:38.948039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:39.876909 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:39.876874 2574 generic.go:356] "Generic (PLEG): container finished" podID="24bd3c43-afed-4a52-a901-c855842b09e8" containerID="c1484f60ffc3cd418492c6b503625cec812982d37018ce009086cb590eec3667" exitCode=0 Jul 14 17:09:39.877333 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:39.876950 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" event={"ID":"24bd3c43-afed-4a52-a901-c855842b09e8","Type":"ContainerDied","Data":"c1484f60ffc3cd418492c6b503625cec812982d37018ce009086cb590eec3667"} Jul 14 17:09:39.878379 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:39.878359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" event={"ID":"ee36e8c5-c5fc-4acf-bf74-2c6123082f8f","Type":"ContainerStarted","Data":"d00071dc80cc8979c44a1da792e3eefff307192e6a70e0e1716b2a22830a2ce3"} Jul 14 17:09:39.878491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:39.878478 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:39.927040 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:39.926946 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" podStartSLOduration=1.037603588 podStartE2EDuration="4.926929403s" totalImagesPullingTime="3.889325815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:09:39.92639059 +0000 UTC m=+273.449331364" watchObservedRunningTime="2026-07-14 17:09:39.926929403 +0000 UTC m=+273.449870175" Jul 14 17:09:41.007444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.007419 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:41.128153 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.128104 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util\") pod \"24bd3c43-afed-4a52-a901-c855842b09e8\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " Jul 14 17:09:41.128320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.128174 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle\") pod \"24bd3c43-afed-4a52-a901-c855842b09e8\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " Jul 14 17:09:41.128320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.128227 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82npv\" (UniqueName: \"kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv\") pod \"24bd3c43-afed-4a52-a901-c855842b09e8\" (UID: \"24bd3c43-afed-4a52-a901-c855842b09e8\") " Jul 14 17:09:41.128618 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.128552 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle" (OuterVolumeSpecName: "bundle") pod "24bd3c43-afed-4a52-a901-c855842b09e8" (UID: "24bd3c43-afed-4a52-a901-c855842b09e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:09:41.130281 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.130256 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv" (OuterVolumeSpecName: "kube-api-access-82npv") pod "24bd3c43-afed-4a52-a901-c855842b09e8" (UID: "24bd3c43-afed-4a52-a901-c855842b09e8"). InnerVolumeSpecName "kube-api-access-82npv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:09:41.132683 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.132629 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util" (OuterVolumeSpecName: "util") pod "24bd3c43-afed-4a52-a901-c855842b09e8" (UID: "24bd3c43-afed-4a52-a901-c855842b09e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:09:41.228800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.228759 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:41.228800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.228795 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82npv\" (UniqueName: \"kubernetes.io/projected/24bd3c43-afed-4a52-a901-c855842b09e8-kube-api-access-82npv\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:41.228800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.228807 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/24bd3c43-afed-4a52-a901-c855842b09e8-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:09:41.887742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.887703 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" event={"ID":"24bd3c43-afed-4a52-a901-c855842b09e8","Type":"ContainerDied","Data":"fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae"} Jul 14 17:09:41.887911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.887751 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fq4vrl" Jul 14 17:09:41.887911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:41.887758 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fffdcf47ce302e1cd2adc4f8691e5220743d09ed5f6607d823fa53ac578b5fae" Jul 14 17:09:44.346093 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:44.346058 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:45.024710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024672 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-grlt9"] Jul 14 17:09:45.024957 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024944 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="util" Jul 14 17:09:45.025000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024959 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="util" Jul 14 17:09:45.025000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024967 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="pull" Jul 14 17:09:45.025000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024972 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="pull" Jul 14 17:09:45.025000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024982 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="extract" Jul 14 17:09:45.025000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.024989 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="extract" Jul 14 17:09:45.025139 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.025040 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="24bd3c43-afed-4a52-a901-c855842b09e8" containerName="extract" Jul 14 17:09:45.029018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.029000 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.031803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.031779 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-j4ckn\"" Jul 14 17:09:45.036609 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.036569 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-grlt9"] Jul 14 17:09:45.162213 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.162178 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-bound-sa-token\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.162380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.162218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vkjq\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-kube-api-access-9vkjq\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.263061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.263020 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-bound-sa-token\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.263235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.263071 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vkjq\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-kube-api-access-9vkjq\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.272944 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.272916 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-bound-sa-token\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.273204 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.273180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vkjq\" (UniqueName: \"kubernetes.io/projected/74b255af-13ea-4cbd-b16a-d7f71187e25f-kube-api-access-9vkjq\") pod \"cert-manager-55b4d954d5-grlt9\" (UID: \"74b255af-13ea-4cbd-b16a-d7f71187e25f\") " pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.337961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.337929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-grlt9" Jul 14 17:09:45.461831 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.461788 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:45.507802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.507769 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-grlt9"] Jul 14 17:09:45.510857 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:45.510825 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74b255af_13ea_4cbd_b16a_d7f71187e25f.slice/crio-6535b6ffa58f6866ca64bfa058b3f176299a9e0e3fe264fb70501036195c5a67 WatchSource:0}: Error finding container 6535b6ffa58f6866ca64bfa058b3f176299a9e0e3fe264fb70501036195c5a67: Status 404 returned error can't find the container with id 6535b6ffa58f6866ca64bfa058b3f176299a9e0e3fe264fb70501036195c5a67 Jul 14 17:09:45.586105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.586068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:45.884444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.884416 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-cjdv7" Jul 14 17:09:45.911989 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.911956 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-grlt9" event={"ID":"74b255af-13ea-4cbd-b16a-d7f71187e25f","Type":"ContainerStarted","Data":"fa3135f347a3671ada1a6ee8cbd33c6835c9cc45522aa5c2ac5f511c94759641"} Jul 14 17:09:45.911989 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.911993 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-grlt9" event={"ID":"74b255af-13ea-4cbd-b16a-d7f71187e25f","Type":"ContainerStarted","Data":"6535b6ffa58f6866ca64bfa058b3f176299a9e0e3fe264fb70501036195c5a67"} Jul 14 17:09:45.939468 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:45.939419 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-grlt9" podStartSLOduration=0.939402747 podStartE2EDuration="939.402747ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:09:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:09:45.938986153 +0000 UTC m=+279.461926929" watchObservedRunningTime="2026-07-14 17:09:45.939402747 +0000 UTC m=+279.462343521" Jul 14 17:09:47.517016 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.516980 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv"] Jul 14 17:09:47.520361 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.520345 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.523936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.523912 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:09:47.524178 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.524161 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 17:09:47.525141 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.525124 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zq85p\"" Jul 14 17:09:47.533198 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.533176 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv"] Jul 14 17:09:47.679297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.679260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfwk9\" (UniqueName: \"kubernetes.io/projected/ca935859-3ba3-4928-9cf0-74d475cf323d-kube-api-access-lfwk9\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.679297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.679299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca935859-3ba3-4928-9cf0-74d475cf323d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.780208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.780114 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfwk9\" (UniqueName: \"kubernetes.io/projected/ca935859-3ba3-4928-9cf0-74d475cf323d-kube-api-access-lfwk9\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.780208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.780153 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca935859-3ba3-4928-9cf0-74d475cf323d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.780544 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.780528 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca935859-3ba3-4928-9cf0-74d475cf323d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.793083 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.793053 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfwk9\" (UniqueName: \"kubernetes.io/projected/ca935859-3ba3-4928-9cf0-74d475cf323d-kube-api-access-lfwk9\") pod \"openshift-lws-operator-fd8ccff4c-6ldrv\" (UID: \"ca935859-3ba3-4928-9cf0-74d475cf323d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.829126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.829091 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" Jul 14 17:09:47.973102 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:47.973066 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv"] Jul 14 17:09:47.976120 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:47.976090 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca935859_3ba3_4928_9cf0_74d475cf323d.slice/crio-3d21b1d778b31b0ec247a8a8cf4b9b8fffa8197ced4e03dfd4c4a9dcb33f1f1d WatchSource:0}: Error finding container 3d21b1d778b31b0ec247a8a8cf4b9b8fffa8197ced4e03dfd4c4a9dcb33f1f1d: Status 404 returned error can't find the container with id 3d21b1d778b31b0ec247a8a8cf4b9b8fffa8197ced4e03dfd4c4a9dcb33f1f1d Jul 14 17:09:48.925704 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:48.925673 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" event={"ID":"ca935859-3ba3-4928-9cf0-74d475cf323d","Type":"ContainerStarted","Data":"3d21b1d778b31b0ec247a8a8cf4b9b8fffa8197ced4e03dfd4c4a9dcb33f1f1d"} Jul 14 17:09:51.940311 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:51.940273 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" event={"ID":"ca935859-3ba3-4928-9cf0-74d475cf323d","Type":"ContainerStarted","Data":"d89c53aed7b7ae83fe9f4731b24e5b1027e2b5d47988ccff33e7a8461cfa7610"} Jul 14 17:09:51.964161 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:51.964106 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6ldrv" podStartSLOduration=2.033359257 podStartE2EDuration="4.964090841s" totalImagesPullingTime="2.930731584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:09:51.963517213 +0000 UTC m=+285.486458002" watchObservedRunningTime="2026-07-14 17:09:51.964090841 +0000 UTC m=+285.487031614" Jul 14 17:09:52.714285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:52.714239 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:52.813590 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:52.813551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:54.877205 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:54.877167 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:09:55.156248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.156147 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw"] Jul 14 17:09:55.161781 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.161325 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.164783 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.164758 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 17:09:55.164909 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.164835 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 17:09:55.165911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.165892 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-j9qc8\"" Jul 14 17:09:55.169922 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.169896 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw"] Jul 14 17:09:55.338371 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.338329 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d679d\" (UniqueName: \"kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.338553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.338393 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.338553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.338438 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.439323 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.439237 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.439323 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.439287 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.439505 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.439341 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d679d\" (UniqueName: \"kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.439704 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.439686 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.439754 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.439738 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.448975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.448941 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d679d\" (UniqueName: \"kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.471813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.471766 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:09:55.604111 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.604077 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw"] Jul 14 17:09:55.607896 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:09:55.607868 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb58b48d_3d28_468c_9cc6_525141314e45.slice/crio-e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255 WatchSource:0}: Error finding container e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255: Status 404 returned error can't find the container with id e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255 Jul 14 17:09:55.957494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.957409 2574 generic.go:356] "Generic (PLEG): container finished" podID="eb58b48d-3d28-468c-9cc6-525141314e45" containerID="bbd14dd83d92d982ed847afed636f8ef2d60385d9de3ad36caf869a35bfa97d1" exitCode=0 Jul 14 17:09:55.957904 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.957496 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" event={"ID":"eb58b48d-3d28-468c-9cc6-525141314e45","Type":"ContainerDied","Data":"bbd14dd83d92d982ed847afed636f8ef2d60385d9de3ad36caf869a35bfa97d1"} Jul 14 17:09:55.957904 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:55.957553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" event={"ID":"eb58b48d-3d28-468c-9cc6-525141314e45","Type":"ContainerStarted","Data":"e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255"} Jul 14 17:09:57.968010 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:57.967981 2574 generic.go:356] "Generic (PLEG): container finished" podID="eb58b48d-3d28-468c-9cc6-525141314e45" containerID="5417d00f40840a1dfd201d78dabb380b65db2705fb7d12905f562afb042218c4" exitCode=0 Jul 14 17:09:57.968450 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:57.968032 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" event={"ID":"eb58b48d-3d28-468c-9cc6-525141314e45","Type":"ContainerDied","Data":"5417d00f40840a1dfd201d78dabb380b65db2705fb7d12905f562afb042218c4"} Jul 14 17:09:58.974159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:58.974122 2574 generic.go:356] "Generic (PLEG): container finished" podID="eb58b48d-3d28-468c-9cc6-525141314e45" containerID="1a93c24574e9c44db25510eb7369adc6589958d9a7e8e183ac4bfc4a44b47c06" exitCode=0 Jul 14 17:09:58.974540 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:09:58.974167 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" event={"ID":"eb58b48d-3d28-468c-9cc6-525141314e45","Type":"ContainerDied","Data":"1a93c24574e9c44db25510eb7369adc6589958d9a7e8e183ac4bfc4a44b47c06"} Jul 14 17:10:00.102445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.102423 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:10:00.170863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.170829 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util\") pod \"eb58b48d-3d28-468c-9cc6-525141314e45\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " Jul 14 17:10:00.171047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.170881 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d679d\" (UniqueName: \"kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d\") pod \"eb58b48d-3d28-468c-9cc6-525141314e45\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " Jul 14 17:10:00.171047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.170905 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle\") pod \"eb58b48d-3d28-468c-9cc6-525141314e45\" (UID: \"eb58b48d-3d28-468c-9cc6-525141314e45\") " Jul 14 17:10:00.171703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.171664 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle" (OuterVolumeSpecName: "bundle") pod "eb58b48d-3d28-468c-9cc6-525141314e45" (UID: "eb58b48d-3d28-468c-9cc6-525141314e45"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:10:00.173073 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.173043 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d" (OuterVolumeSpecName: "kube-api-access-d679d") pod "eb58b48d-3d28-468c-9cc6-525141314e45" (UID: "eb58b48d-3d28-468c-9cc6-525141314e45"). InnerVolumeSpecName "kube-api-access-d679d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:10:00.176619 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.176593 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util" (OuterVolumeSpecName: "util") pod "eb58b48d-3d28-468c-9cc6-525141314e45" (UID: "eb58b48d-3d28-468c-9cc6-525141314e45"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:10:00.271507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.271419 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d679d\" (UniqueName: \"kubernetes.io/projected/eb58b48d-3d28-468c-9cc6-525141314e45-kube-api-access-d679d\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:10:00.271507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.271450 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:10:00.271507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.271460 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb58b48d-3d28-468c-9cc6-525141314e45-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:10:00.984334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.984298 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" event={"ID":"eb58b48d-3d28-468c-9cc6-525141314e45","Type":"ContainerDied","Data":"e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255"} Jul 14 17:10:00.984334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.984327 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af56lrw" Jul 14 17:10:00.984546 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:00.984347 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1399032109cb2a006667a3d892ea59664aa235965e54953df45b22fa06db255" Jul 14 17:10:06.915942 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:06.915897 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:07.536552 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.536526 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:10:07.538023 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.538000 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:10:07.539946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.539924 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 17:10:07.934414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934385 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf"] Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934676 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="util" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934688 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="util" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934703 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="pull" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934708 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="pull" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934722 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="extract" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934727 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="extract" Jul 14 17:10:07.936615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.934775 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb58b48d-3d28-468c-9cc6-525141314e45" containerName="extract" Jul 14 17:10:07.937494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.937476 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:07.940460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.940426 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 17:10:07.940624 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.940512 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 17:10:07.940719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.940701 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fv67k\"" Jul 14 17:10:07.940799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.940781 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 17:10:07.940854 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.940795 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 17:10:07.954021 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:07.953949 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf"] Jul 14 17:10:08.032756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.032713 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.032756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.032759 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.032988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.032788 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.032988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.032869 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.032988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.032907 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl9q4\" (UniqueName: \"kubernetes.io/projected/3cfb5224-e6bc-453b-8f47-e0a14a13900e-kube-api-access-rl9q4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134117 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134073 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134117 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134146 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134200 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl9q4\" (UniqueName: \"kubernetes.io/projected/3cfb5224-e6bc-453b-8f47-e0a14a13900e-kube-api-access-rl9q4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.134613 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.134569 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-manifests\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.136435 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.136414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3cfb5224-e6bc-453b-8f47-e0a14a13900e-tmp\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.136679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.136652 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.136815 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.136795 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3cfb5224-e6bc-453b-8f47-e0a14a13900e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.147269 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.147235 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl9q4\" (UniqueName: \"kubernetes.io/projected/3cfb5224-e6bc-453b-8f47-e0a14a13900e-kube-api-access-rl9q4\") pod \"opendatahub-operator-controller-manager-7655f47d9d-w8tdf\" (UID: \"3cfb5224-e6bc-453b-8f47-e0a14a13900e\") " pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.249216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.249118 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:08.401549 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.401514 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf"] Jul 14 17:10:08.404737 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:10:08.404708 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfb5224_e6bc_453b_8f47_e0a14a13900e.slice/crio-c011505d61633fc5b59b346633e4e1c839aa3476be79463b35a01a0143622a21 WatchSource:0}: Error finding container c011505d61633fc5b59b346633e4e1c839aa3476be79463b35a01a0143622a21: Status 404 returned error can't find the container with id c011505d61633fc5b59b346633e4e1c839aa3476be79463b35a01a0143622a21 Jul 14 17:10:08.771399 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:08.771358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:09.018859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:09.018826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" event={"ID":"3cfb5224-e6bc-453b-8f47-e0a14a13900e","Type":"ContainerStarted","Data":"c011505d61633fc5b59b346633e4e1c839aa3476be79463b35a01a0143622a21"} Jul 14 17:10:12.035829 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:12.035744 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" event={"ID":"3cfb5224-e6bc-453b-8f47-e0a14a13900e","Type":"ContainerStarted","Data":"3822dd09166b0b8ed830a2b9baf1fdd0efcf24b62cd9c4fd3204640ac9f3a50a"} Jul 14 17:10:13.041887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:13.041852 2574 generic.go:356] "Generic (PLEG): container finished" podID="3cfb5224-e6bc-453b-8f47-e0a14a13900e" containerID="3822dd09166b0b8ed830a2b9baf1fdd0efcf24b62cd9c4fd3204640ac9f3a50a" exitCode=0 Jul 14 17:10:13.042341 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:13.041941 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" event={"ID":"3cfb5224-e6bc-453b-8f47-e0a14a13900e","Type":"ContainerDied","Data":"3822dd09166b0b8ed830a2b9baf1fdd0efcf24b62cd9c4fd3204640ac9f3a50a"} Jul 14 17:10:13.042512 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:13.042494 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:10:14.048290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:14.048253 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" event={"ID":"3cfb5224-e6bc-453b-8f47-e0a14a13900e","Type":"ContainerStarted","Data":"cdf9605fd96dc859313aacb7a102514ba8960556d640ec42bfceda9165be95fa"} Jul 14 17:10:14.048789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:14.048339 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:14.092813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:14.092757 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" podStartSLOduration=2.25244328 podStartE2EDuration="7.092738431s" totalImagesPullingTime="3.667737364s" totalInitContainerRuntime="1.172557787s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:14.091077077 +0000 UTC m=+307.614017863" watchObservedRunningTime="2026-07-14 17:10:14.092738431 +0000 UTC m=+307.615679204" Jul 14 17:10:17.157267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:17.157215 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:17.282599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:17.282544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:25.054950 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:25.054919 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7655f47d9d-w8tdf" Jul 14 17:10:26.666283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:26.666227 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:26.762669 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:26.762636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.363995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.363958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.448651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.448611 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5"] Jul 14 17:10:27.455117 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.455086 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.457982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.457955 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 17:10:27.457982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.457972 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 17:10:27.458417 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.458391 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 17:10:27.459141 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.459121 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-645gb\"" Jul 14 17:10:27.459241 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.459140 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 17:10:27.469094 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.469066 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5"] Jul 14 17:10:27.492313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.492279 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcxb\" (UniqueName: \"kubernetes.io/projected/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-kube-api-access-rbcxb\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.492466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.492327 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.492466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.492417 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tmp\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.556328 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.556297 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.592886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.592837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcxb\" (UniqueName: \"kubernetes.io/projected/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-kube-api-access-rbcxb\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.592886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.592891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.593147 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.592920 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tmp\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.595704 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.595678 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tmp\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.596083 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.596058 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-tls-certs\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.602454 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.602423 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcxb\" (UniqueName: \"kubernetes.io/projected/a2bfe160-ae19-4df2-90d2-de4e7a8e28ab-kube-api-access-rbcxb\") pod \"kube-auth-proxy-5c548b7c75-frzt5\" (UID: \"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab\") " pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.618542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.618448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.722468 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.722427 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.766303 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.766267 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" Jul 14 17:10:27.826511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.826482 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:27.919744 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.919711 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5"] Jul 14 17:10:27.921281 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:10:27.921251 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2bfe160_ae19_4df2_90d2_de4e7a8e28ab.slice/crio-2c00cea4588d3183affdf459bd03d548b7dfe70b6398989a4940fc4e53bcb1fc WatchSource:0}: Error finding container 2c00cea4588d3183affdf459bd03d548b7dfe70b6398989a4940fc4e53bcb1fc: Status 404 returned error can't find the container with id 2c00cea4588d3183affdf459bd03d548b7dfe70b6398989a4940fc4e53bcb1fc Jul 14 17:10:27.925374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:27.925340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:28.116729 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:28.116693 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" event={"ID":"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab","Type":"ContainerStarted","Data":"2c00cea4588d3183affdf459bd03d548b7dfe70b6398989a4940fc4e53bcb1fc"} Jul 14 17:10:28.473020 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:28.472975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:28.625172 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:28.625135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:28.720377 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:28.720335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:28.819919 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:28.819829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:29.853708 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:29.853671 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:33.140530 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:33.140496 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" event={"ID":"a2bfe160-ae19-4df2-90d2-de4e7a8e28ab","Type":"ContainerStarted","Data":"8c3a62f8501ca9ff6506e14650fa70c29b34dae8abd40e7918db5e35518a1fe2"} Jul 14 17:10:33.160744 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:33.160697 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c548b7c75-frzt5" podStartSLOduration=1.25660776 podStartE2EDuration="6.16068176s" totalImagesPullingTime="4.904074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:33.159233317 +0000 UTC m=+326.682174090" watchObservedRunningTime="2026-07-14 17:10:33.16068176 +0000 UTC m=+326.683622533" Jul 14 17:10:34.080936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.080902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:34.230602 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.230550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:34.354638 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.354543 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9"] Jul 14 17:10:34.357859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.357836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.360858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.360839 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 17:10:34.360962 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.360856 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-br89z\"" Jul 14 17:10:34.377611 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.377565 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9"] Jul 14 17:10:34.419623 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.419585 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:34.452650 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452650 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452657 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452677 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452692 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452776 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvl6x\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-kube-api-access-vvl6x\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452818 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452846 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.452875 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452867 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.453085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.452883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553522 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553489 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553529 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553625 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvl6x\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-kube-api-access-vvl6x\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553682 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553717 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.553913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553763 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.554140 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.553968 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.554140 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.554094 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.554386 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.554363 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.554691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.554670 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.554793 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.554776 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.556265 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.556245 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.556762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.556739 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.575946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.575911 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.576375 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.576349 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvl6x\" (UniqueName: \"kubernetes.io/projected/c5f7dfcd-8c5e-4eab-8ced-e2f172da05be-kube-api-access-vvl6x\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4zmsd9\" (UID: \"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.610284 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.610140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:34.670711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.670678 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:34.715013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.714928 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:34.820620 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:34.817045 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9"] Jul 14 17:10:35.153814 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.153769 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" event={"ID":"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be","Type":"ContainerStarted","Data":"5f062308c15c75ee2ba975aebb8d3862236353fd901f436a3464bbbdfa5ac79a"} Jul 14 17:10:35.484919 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.484843 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-djz47"] Jul 14 17:10:35.489210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.489191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.493248 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.493225 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jxjns\"" Jul 14 17:10:35.493360 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.493337 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 17:10:35.493427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.493385 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 17:10:35.493488 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.493441 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 17:10:35.498644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.498612 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-djz47"] Jul 14 17:10:35.563728 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.563684 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29fcdfcb-3e96-4afa-a416-99fcafef228c-manager-config\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.563906 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.563779 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbttw\" (UniqueName: \"kubernetes.io/projected/29fcdfcb-3e96-4afa-a416-99fcafef228c-kube-api-access-jbttw\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.563906 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.563858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.564032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.563917 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-metrics-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.664479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.664442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.664479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.664495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-metrics-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.664794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.664762 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29fcdfcb-3e96-4afa-a416-99fcafef228c-manager-config\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.664964 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.664837 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbttw\" (UniqueName: \"kubernetes.io/projected/29fcdfcb-3e96-4afa-a416-99fcafef228c-kube-api-access-jbttw\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.665703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.665676 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29fcdfcb-3e96-4afa-a416-99fcafef228c-manager-config\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.667674 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.667641 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.667769 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.667645 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/29fcdfcb-3e96-4afa-a416-99fcafef228c-metrics-cert\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.675411 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.675086 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbttw\" (UniqueName: \"kubernetes.io/projected/29fcdfcb-3e96-4afa-a416-99fcafef228c-kube-api-access-jbttw\") pod \"lws-controller-manager-5684c55548-djz47\" (UID: \"29fcdfcb-3e96-4afa-a416-99fcafef228c\") " pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:35.799680 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:35.799597 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:36.015937 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:36.015812 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5684c55548-djz47"] Jul 14 17:10:36.018064 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:10:36.018035 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fcdfcb_3e96_4afa_a416_99fcafef228c.slice/crio-d0ba9d4f5b609ba0647d1be913b16d450bd6e07fcb67450e8bb3d06face0ace7 WatchSource:0}: Error finding container d0ba9d4f5b609ba0647d1be913b16d450bd6e07fcb67450e8bb3d06face0ace7: Status 404 returned error can't find the container with id d0ba9d4f5b609ba0647d1be913b16d450bd6e07fcb67450e8bb3d06face0ace7 Jul 14 17:10:36.159095 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:36.159061 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" event={"ID":"29fcdfcb-3e96-4afa-a416-99fcafef228c","Type":"ContainerStarted","Data":"d0ba9d4f5b609ba0647d1be913b16d450bd6e07fcb67450e8bb3d06face0ace7"} Jul 14 17:10:38.071407 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.071359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:38.172918 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.172881 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:38.276145 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.276105 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:38.509137 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.509096 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:38.670412 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.670363 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:10:38.670521 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.670477 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:10:38.670593 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:38.670522 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:10:39.179531 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.179480 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" event={"ID":"c5f7dfcd-8c5e-4eab-8ced-e2f172da05be","Type":"ContainerStarted","Data":"5021a801ab1cd8d2762b9cb297f937ea38d4fa42cbb6616939c52637a854351b"} Jul 14 17:10:39.181176 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.181150 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" event={"ID":"29fcdfcb-3e96-4afa-a416-99fcafef228c","Type":"ContainerStarted","Data":"051d0fc853b6544e5f33ad93b263cede7ff70286e5398c1b514a15b3ed7a2b30"} Jul 14 17:10:39.181316 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.181299 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:10:39.207632 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.207553 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" podStartSLOduration=1.359889768 podStartE2EDuration="5.207538346s" totalImagesPullingTime="3.847648578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:39.206327381 +0000 UTC m=+332.729268154" watchObservedRunningTime="2026-07-14 17:10:39.207538346 +0000 UTC m=+332.730479123" Jul 14 17:10:39.236409 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.236354 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" podStartSLOduration=1.5860245819999998 podStartE2EDuration="4.236336473s" totalImagesPullingTime="2.650311891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:10:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:10:39.235339935 +0000 UTC m=+332.758280710" watchObservedRunningTime="2026-07-14 17:10:39.236336473 +0000 UTC m=+332.759277245" Jul 14 17:10:39.641428 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.641386 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:39.671224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.671191 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:39.676038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.676012 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:39.729079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.729039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:39.796786 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:39.796739 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:10:40.185472 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:40.185441 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:40.186414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:40.186393 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4zmsd9" Jul 14 17:10:50.187476 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:10:50.187443 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5684c55548-djz47" Jul 14 17:11:24.063970 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:24.063883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:24.319741 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:24.319651 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:25.778771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:25.778726 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:25.851068 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:25.851035 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:28.679235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:28.679204 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:28.770889 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:28.770857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:28.858685 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:28.858646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:42.267169 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:42.267135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:42.343197 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:42.343161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:42.411957 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:42.411925 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:42.986544 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:42.986511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:43.048559 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:43.048518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:44.159565 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.159533 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9"] Jul 14 17:11:44.162184 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.162162 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.164800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.164781 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:11:44.165800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.165782 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:11:44.165890 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.165821 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-25jch\"" Jul 14 17:11:44.172749 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.172724 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9"] Jul 14 17:11:44.243476 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.243431 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr8kg\" (UniqueName: \"kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.243476 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.243476 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.243727 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.243509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.344028 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.343987 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.344247 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.344053 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.344247 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.344140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr8kg\" (UniqueName: \"kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.344444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.344424 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.344503 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.344487 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.353615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.353591 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr8kg\" (UniqueName: \"kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.471857 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.471772 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:44.606333 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.606306 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9"] Jul 14 17:11:44.608415 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:11:44.608385 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5e81c1_2550_4aaa_a881_e146827f07c0.slice/crio-bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5 WatchSource:0}: Error finding container bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5: Status 404 returned error can't find the container with id bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5 Jul 14 17:11:44.757160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.757076 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24"] Jul 14 17:11:44.759422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.759400 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.769374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.769339 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24"] Jul 14 17:11:44.847514 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.847480 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.847514 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.847518 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-995vh\" (UniqueName: \"kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.847753 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.847557 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.948777 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.948740 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.948937 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.948834 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.948937 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.948859 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-995vh\" (UniqueName: \"kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.949123 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.949106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.949212 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.949195 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:44.959234 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:44.959210 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-995vh\" (UniqueName: \"kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:45.097759 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.097727 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:45.157939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.157909 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk"] Jul 14 17:11:45.160834 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.160803 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.170053 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.170022 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk"] Jul 14 17:11:45.238188 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.238160 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24"] Jul 14 17:11:45.241286 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:11:45.241251 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578685c7_a6d8_401d_9bc8_d45ebed7d344.slice/crio-b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc WatchSource:0}: Error finding container b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc: Status 404 returned error can't find the container with id b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc Jul 14 17:11:45.251622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.251599 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.251707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.251652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.251707 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.251672 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zw65\" (UniqueName: \"kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.352730 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.352647 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.352730 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.352689 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zw65\" (UniqueName: \"kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.352920 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.352748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.353045 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.353028 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.353086 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.353074 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.362967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.362935 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zw65\" (UniqueName: \"kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.471678 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.471645 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:45.474859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.474832 2574 generic.go:356] "Generic (PLEG): container finished" podID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerID="2ecbbaa9968fcbb073e3c0c0b8a94a1b9eab6df3df14b8f079fcbe33ea75b820" exitCode=0 Jul 14 17:11:45.474958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.474923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" event={"ID":"2d5e81c1-2550-4aaa-a881-e146827f07c0","Type":"ContainerDied","Data":"2ecbbaa9968fcbb073e3c0c0b8a94a1b9eab6df3df14b8f079fcbe33ea75b820"} Jul 14 17:11:45.475018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.474978 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" event={"ID":"2d5e81c1-2550-4aaa-a881-e146827f07c0","Type":"ContainerStarted","Data":"bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5"} Jul 14 17:11:45.476820 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.476794 2574 generic.go:356] "Generic (PLEG): container finished" podID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerID="079d4070377a107b15300210ab45370a8cef54176bbebd46bf7671dd4bfcb3bd" exitCode=0 Jul 14 17:11:45.476917 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.476831 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" event={"ID":"578685c7-a6d8-401d-9bc8-d45ebed7d344","Type":"ContainerDied","Data":"079d4070377a107b15300210ab45370a8cef54176bbebd46bf7671dd4bfcb3bd"} Jul 14 17:11:45.476917 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.476910 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" event={"ID":"578685c7-a6d8-401d-9bc8-d45ebed7d344","Type":"ContainerStarted","Data":"b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc"} Jul 14 17:11:45.556609 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.556555 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf"] Jul 14 17:11:45.560347 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.560323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.568523 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.568491 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf"] Jul 14 17:11:45.612188 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.612089 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk"] Jul 14 17:11:45.617674 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:11:45.617639 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8bbfd0a_e37e_481b_879e_d8b898ad5599.slice/crio-e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261 WatchSource:0}: Error finding container e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261: Status 404 returned error can't find the container with id e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261 Jul 14 17:11:45.655672 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.655613 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxsgm\" (UniqueName: \"kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.655832 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.655766 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.655832 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.655810 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.756542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.756501 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxsgm\" (UniqueName: \"kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.756745 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.756683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.756745 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.756715 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.757105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.757082 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.757174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.757109 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.766267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.766237 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxsgm\" (UniqueName: \"kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:45.872858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:45.872767 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:46.005491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.005352 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf"] Jul 14 17:11:46.008052 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:11:46.008015 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff7519d1_65c8_4936_b255_a4def886833c.slice/crio-b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4 WatchSource:0}: Error finding container b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4: Status 404 returned error can't find the container with id b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4 Jul 14 17:11:46.484789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.484756 2574 generic.go:356] "Generic (PLEG): container finished" podID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerID="bf08bef6cbf0e3be1fb0e1a083fa2fc963d1bebd3409eb2de966764ae49ae192" exitCode=0 Jul 14 17:11:46.485185 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.484834 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" event={"ID":"578685c7-a6d8-401d-9bc8-d45ebed7d344","Type":"ContainerDied","Data":"bf08bef6cbf0e3be1fb0e1a083fa2fc963d1bebd3409eb2de966764ae49ae192"} Jul 14 17:11:46.486382 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.486358 2574 generic.go:356] "Generic (PLEG): container finished" podID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerID="52919eeeddf8dff3729682b7c20be7dbd906e7cfff9da3f30a1d32822092f9b2" exitCode=0 Jul 14 17:11:46.486448 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.486421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" event={"ID":"b8bbfd0a-e37e-481b-879e-d8b898ad5599","Type":"ContainerDied","Data":"52919eeeddf8dff3729682b7c20be7dbd906e7cfff9da3f30a1d32822092f9b2"} Jul 14 17:11:46.486501 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.486474 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" event={"ID":"b8bbfd0a-e37e-481b-879e-d8b898ad5599","Type":"ContainerStarted","Data":"e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261"} Jul 14 17:11:46.488323 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.488290 2574 generic.go:356] "Generic (PLEG): container finished" podID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerID="e42f6cedc7558259a6ffd27dc8a71327aa49ebbcdbe034321003923fafb9f823" exitCode=0 Jul 14 17:11:46.488447 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.488329 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" event={"ID":"2d5e81c1-2550-4aaa-a881-e146827f07c0","Type":"ContainerDied","Data":"e42f6cedc7558259a6ffd27dc8a71327aa49ebbcdbe034321003923fafb9f823"} Jul 14 17:11:46.490314 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.490293 2574 generic.go:356] "Generic (PLEG): container finished" podID="ff7519d1-65c8-4936-b255-a4def886833c" containerID="c917e898a0e3c24ad4d219bc36c6171bed5ac6c1acd1ee28dcca41b032457c8e" exitCode=0 Jul 14 17:11:46.490397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.490327 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" event={"ID":"ff7519d1-65c8-4936-b255-a4def886833c","Type":"ContainerDied","Data":"c917e898a0e3c24ad4d219bc36c6171bed5ac6c1acd1ee28dcca41b032457c8e"} Jul 14 17:11:46.490397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:46.490356 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" event={"ID":"ff7519d1-65c8-4936-b255-a4def886833c","Type":"ContainerStarted","Data":"b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4"} Jul 14 17:11:47.497643 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.497550 2574 generic.go:356] "Generic (PLEG): container finished" podID="ff7519d1-65c8-4936-b255-a4def886833c" containerID="9b66ee47275e29f0ed44543241754ca78f595063f5242d18083fcef770e2d1f6" exitCode=0 Jul 14 17:11:47.498051 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.497637 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" event={"ID":"ff7519d1-65c8-4936-b255-a4def886833c","Type":"ContainerDied","Data":"9b66ee47275e29f0ed44543241754ca78f595063f5242d18083fcef770e2d1f6"} Jul 14 17:11:47.499841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.499819 2574 generic.go:356] "Generic (PLEG): container finished" podID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerID="cfd40995a010f43c2328f3485fc7efdd083175c176c0750ac901c07ac0a7b783" exitCode=0 Jul 14 17:11:47.499946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.499891 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" event={"ID":"578685c7-a6d8-401d-9bc8-d45ebed7d344","Type":"ContainerDied","Data":"cfd40995a010f43c2328f3485fc7efdd083175c176c0750ac901c07ac0a7b783"} Jul 14 17:11:47.501600 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.501530 2574 generic.go:356] "Generic (PLEG): container finished" podID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerID="f70a4591a3b2ef693641152f9a2f217b123df77c30a4c1753dff6b0646d4d46d" exitCode=0 Jul 14 17:11:47.501687 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.501597 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" event={"ID":"b8bbfd0a-e37e-481b-879e-d8b898ad5599","Type":"ContainerDied","Data":"f70a4591a3b2ef693641152f9a2f217b123df77c30a4c1753dff6b0646d4d46d"} Jul 14 17:11:47.503816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.503793 2574 generic.go:356] "Generic (PLEG): container finished" podID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerID="97eb6751ab04b66ab36a2350127f265f1d23926823807884712966b61a27ad4a" exitCode=0 Jul 14 17:11:47.503896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:47.503846 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" event={"ID":"2d5e81c1-2550-4aaa-a881-e146827f07c0","Type":"ContainerDied","Data":"97eb6751ab04b66ab36a2350127f265f1d23926823807884712966b61a27ad4a"} Jul 14 17:11:48.510774 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.510733 2574 generic.go:356] "Generic (PLEG): container finished" podID="ff7519d1-65c8-4936-b255-a4def886833c" containerID="70bc449be1f8d404b2f4e8181a94ac397b6553ebcec967679bb64d4d229d0b67" exitCode=0 Jul 14 17:11:48.511170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.510819 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" event={"ID":"ff7519d1-65c8-4936-b255-a4def886833c","Type":"ContainerDied","Data":"70bc449be1f8d404b2f4e8181a94ac397b6553ebcec967679bb64d4d229d0b67"} Jul 14 17:11:48.512744 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.512718 2574 generic.go:356] "Generic (PLEG): container finished" podID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerID="a2fa6255779e28a0ef5733dd1a1f8b447323a5bec7ee3d9b80f313d2d37f9dbb" exitCode=0 Jul 14 17:11:48.512848 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.512805 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" event={"ID":"b8bbfd0a-e37e-481b-879e-d8b898ad5599","Type":"ContainerDied","Data":"a2fa6255779e28a0ef5733dd1a1f8b447323a5bec7ee3d9b80f313d2d37f9dbb"} Jul 14 17:11:48.647106 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.647081 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:48.677376 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.677350 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:48.779373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779277 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util\") pod \"578685c7-a6d8-401d-9bc8-d45ebed7d344\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " Jul 14 17:11:48.779373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779340 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-995vh\" (UniqueName: \"kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh\") pod \"578685c7-a6d8-401d-9bc8-d45ebed7d344\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " Jul 14 17:11:48.779644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779394 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util\") pod \"2d5e81c1-2550-4aaa-a881-e146827f07c0\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " Jul 14 17:11:48.779644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779427 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr8kg\" (UniqueName: \"kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg\") pod \"2d5e81c1-2550-4aaa-a881-e146827f07c0\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " Jul 14 17:11:48.779644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779481 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle\") pod \"2d5e81c1-2550-4aaa-a881-e146827f07c0\" (UID: \"2d5e81c1-2550-4aaa-a881-e146827f07c0\") " Jul 14 17:11:48.779644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.779527 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle\") pod \"578685c7-a6d8-401d-9bc8-d45ebed7d344\" (UID: \"578685c7-a6d8-401d-9bc8-d45ebed7d344\") " Jul 14 17:11:48.780112 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.780080 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle" (OuterVolumeSpecName: "bundle") pod "2d5e81c1-2550-4aaa-a881-e146827f07c0" (UID: "2d5e81c1-2550-4aaa-a881-e146827f07c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:48.780277 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.780249 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle" (OuterVolumeSpecName: "bundle") pod "578685c7-a6d8-401d-9bc8-d45ebed7d344" (UID: "578685c7-a6d8-401d-9bc8-d45ebed7d344"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:48.781801 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.781774 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh" (OuterVolumeSpecName: "kube-api-access-995vh") pod "578685c7-a6d8-401d-9bc8-d45ebed7d344" (UID: "578685c7-a6d8-401d-9bc8-d45ebed7d344"). InnerVolumeSpecName "kube-api-access-995vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:11:48.781897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.781872 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg" (OuterVolumeSpecName: "kube-api-access-sr8kg") pod "2d5e81c1-2550-4aaa-a881-e146827f07c0" (UID: "2d5e81c1-2550-4aaa-a881-e146827f07c0"). InnerVolumeSpecName "kube-api-access-sr8kg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:11:48.785121 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.785084 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util" (OuterVolumeSpecName: "util") pod "578685c7-a6d8-401d-9bc8-d45ebed7d344" (UID: "578685c7-a6d8-401d-9bc8-d45ebed7d344"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:48.785558 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.785540 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util" (OuterVolumeSpecName: "util") pod "2d5e81c1-2550-4aaa-a881-e146827f07c0" (UID: "2d5e81c1-2550-4aaa-a881-e146827f07c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:48.880803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880760 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:48.880803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880796 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:48.880803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880804 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/578685c7-a6d8-401d-9bc8-d45ebed7d344-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:48.881027 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880814 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-995vh\" (UniqueName: \"kubernetes.io/projected/578685c7-a6d8-401d-9bc8-d45ebed7d344-kube-api-access-995vh\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:48.881027 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880824 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2d5e81c1-2550-4aaa-a881-e146827f07c0-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:48.881027 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:48.880833 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr8kg\" (UniqueName: \"kubernetes.io/projected/2d5e81c1-2550-4aaa-a881-e146827f07c0-kube-api-access-sr8kg\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.519260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.519230 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" Jul 14 17:11:49.519260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.519252 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24" event={"ID":"578685c7-a6d8-401d-9bc8-d45ebed7d344","Type":"ContainerDied","Data":"b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc"} Jul 14 17:11:49.519727 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.519293 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5b54d8460c7c6fc1a49e34eb36714505519a591d8f1e97e4e6c18ffae3c82dc" Jul 14 17:11:49.521130 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.521095 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" event={"ID":"2d5e81c1-2550-4aaa-a881-e146827f07c0","Type":"ContainerDied","Data":"bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5"} Jul 14 17:11:49.521130 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.521117 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9" Jul 14 17:11:49.521283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.521145 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bcf17c1c2517a0ccbca57f52e006707efc7dd545dec29989b9f85ab2c91392e5" Jul 14 17:11:49.650712 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.650685 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:49.684435 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.684408 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:49.788840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788746 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util\") pod \"ff7519d1-65c8-4936-b255-a4def886833c\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " Jul 14 17:11:49.788840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788805 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle\") pod \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " Jul 14 17:11:49.788840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788822 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle\") pod \"ff7519d1-65c8-4936-b255-a4def886833c\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " Jul 14 17:11:49.789120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788862 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxsgm\" (UniqueName: \"kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm\") pod \"ff7519d1-65c8-4936-b255-a4def886833c\" (UID: \"ff7519d1-65c8-4936-b255-a4def886833c\") " Jul 14 17:11:49.789120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788889 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util\") pod \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " Jul 14 17:11:49.789120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.788935 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zw65\" (UniqueName: \"kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65\") pod \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\" (UID: \"b8bbfd0a-e37e-481b-879e-d8b898ad5599\") " Jul 14 17:11:49.789410 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.789380 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle" (OuterVolumeSpecName: "bundle") pod "ff7519d1-65c8-4936-b255-a4def886833c" (UID: "ff7519d1-65c8-4936-b255-a4def886833c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:49.789516 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.789410 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle" (OuterVolumeSpecName: "bundle") pod "b8bbfd0a-e37e-481b-879e-d8b898ad5599" (UID: "b8bbfd0a-e37e-481b-879e-d8b898ad5599"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:49.789654 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.789528 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.789654 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.789548 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.791338 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.791314 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm" (OuterVolumeSpecName: "kube-api-access-zxsgm") pod "ff7519d1-65c8-4936-b255-a4def886833c" (UID: "ff7519d1-65c8-4936-b255-a4def886833c"). InnerVolumeSpecName "kube-api-access-zxsgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:11:49.791457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.791436 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65" (OuterVolumeSpecName: "kube-api-access-9zw65") pod "b8bbfd0a-e37e-481b-879e-d8b898ad5599" (UID: "b8bbfd0a-e37e-481b-879e-d8b898ad5599"). InnerVolumeSpecName "kube-api-access-9zw65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:11:49.797985 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.797948 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util" (OuterVolumeSpecName: "util") pod "ff7519d1-65c8-4936-b255-a4def886833c" (UID: "ff7519d1-65c8-4936-b255-a4def886833c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:49.798168 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.798150 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util" (OuterVolumeSpecName: "util") pod "b8bbfd0a-e37e-481b-879e-d8b898ad5599" (UID: "b8bbfd0a-e37e-481b-879e-d8b898ad5599"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:11:49.890766 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.890709 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxsgm\" (UniqueName: \"kubernetes.io/projected/ff7519d1-65c8-4936-b255-a4def886833c-kube-api-access-zxsgm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.890766 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.890757 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8bbfd0a-e37e-481b-879e-d8b898ad5599-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.890766 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.890768 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zw65\" (UniqueName: \"kubernetes.io/projected/b8bbfd0a-e37e-481b-879e-d8b898ad5599-kube-api-access-9zw65\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:49.890766 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:49.890778 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff7519d1-65c8-4936-b255-a4def886833c-util\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:11:50.528245 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.528211 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" Jul 14 17:11:50.528245 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.528226 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk" event={"ID":"b8bbfd0a-e37e-481b-879e-d8b898ad5599","Type":"ContainerDied","Data":"e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261"} Jul 14 17:11:50.528728 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.528277 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e73c16f6d3834a0eee1a2ad1f39b3ead346b22e1645545e996cb0f8b77560261" Jul 14 17:11:50.530061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.530041 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" event={"ID":"ff7519d1-65c8-4936-b255-a4def886833c","Type":"ContainerDied","Data":"b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4"} Jul 14 17:11:50.530154 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.530073 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b57bd7ae9dbb6fffa207471698dd05b7013094ea1b36246622ea9cd8c17635f4" Jul 14 17:11:50.530154 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:50.530073 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf" Jul 14 17:11:51.706511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:51.706463 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:52.589079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:52.589037 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:11:52.717285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:52.717241 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:52.910497 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:52.910391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:53.010341 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:53.010304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:55.090009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:55.089973 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:57.055465 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:57.055425 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:59.095168 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:59.095133 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:11:59.259326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:11:59.259294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:01.902311 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:01.902278 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:02.072067 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:02.072034 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:02.866647 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:02.866609 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:04.342489 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:04.342452 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:04.444539 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:04.444504 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:07.268339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:07.268308 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:07.328056 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:07.328021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:08.317483 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:08.317447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:08.391113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:08.391076 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:08.631046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:08.631009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:08.699043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:08.699007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:14.943751 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:14.943711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:15.043445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:15.043408 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:17.609553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.609513 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-797c458578-f4trg" podUID="bb2c9056-5205-457c-86cf-cccd6a8891e1" containerName="console" containerID="cri-o://b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2" gracePeriod=15 Jul 14 17:12:17.854706 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.854683 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:12:17.922440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922353 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkc8z\" (UniqueName: \"kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922398 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922428 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922544 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922642 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922679 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922725 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca\") pod \"bb2c9056-5205-457c-86cf-cccd6a8891e1\" (UID: \"bb2c9056-5205-457c-86cf-cccd6a8891e1\") " Jul 14 17:12:17.922966 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.922932 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:12:17.923263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.923221 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:12:17.923814 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.923660 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config" (OuterVolumeSpecName: "console-config") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:12:17.923814 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.923670 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca" (OuterVolumeSpecName: "service-ca") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:12:17.924907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.924881 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:12:17.924907 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.924889 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z" (OuterVolumeSpecName: "kube-api-access-qkc8z") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "kube-api-access-qkc8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:12:17.925170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:17.925135 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bb2c9056-5205-457c-86cf-cccd6a8891e1" (UID: "bb2c9056-5205-457c-86cf-cccd6a8891e1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:12:18.023691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023656 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-trusted-ca-bundle\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023685 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023691 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023696 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-service-ca\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023935 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023704 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkc8z\" (UniqueName: \"kubernetes.io/projected/bb2c9056-5205-457c-86cf-cccd6a8891e1-kube-api-access-qkc8z\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023935 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023714 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023935 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023722 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bb2c9056-5205-457c-86cf-cccd6a8891e1-console-oauth-config\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.023935 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.023730 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bb2c9056-5205-457c-86cf-cccd6a8891e1-oauth-serving-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:12:18.663285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.663248 2574 generic.go:356] "Generic (PLEG): container finished" podID="bb2c9056-5205-457c-86cf-cccd6a8891e1" containerID="b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2" exitCode=0 Jul 14 17:12:18.663778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.663297 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797c458578-f4trg" Jul 14 17:12:18.663778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.663327 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c458578-f4trg" event={"ID":"bb2c9056-5205-457c-86cf-cccd6a8891e1","Type":"ContainerDied","Data":"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2"} Jul 14 17:12:18.663778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.663381 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797c458578-f4trg" event={"ID":"bb2c9056-5205-457c-86cf-cccd6a8891e1","Type":"ContainerDied","Data":"cd5b477d969bac07a2ae04e124555eae7332a611a338582b8072b9f7b3ed8b02"} Jul 14 17:12:18.663778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.663401 2574 scope.go:117] "RemoveContainer" containerID="b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2" Jul 14 17:12:18.673639 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.673615 2574 scope.go:117] "RemoveContainer" containerID="b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2" Jul 14 17:12:18.673984 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:12:18.673959 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2\": container with ID starting with b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2 not found: ID does not exist" containerID="b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2" Jul 14 17:12:18.674043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.674001 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2"} err="failed to get container status \"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2\": rpc error: code = NotFound desc = could not find container \"b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2\": container with ID starting with b35f31c55142e4c0b600519d7e25704202125b4b79b95f64692397f4be4f95b2 not found: ID does not exist" Jul 14 17:12:18.710014 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.709984 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:12:18.712782 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:18.712757 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-797c458578-f4trg"] Jul 14 17:12:19.386981 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:19.386941 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:19.692167 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:19.692090 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb2c9056-5205-457c-86cf-cccd6a8891e1" path="/var/lib/kubelet/pods/bb2c9056-5205-457c-86cf-cccd6a8891e1/volumes" Jul 14 17:12:19.708274 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:19.708242 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:19.816205 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:19.816162 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:25.042950 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:25.042911 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:25.142346 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:25.142312 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:28.499835 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:28.499749 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:28.574759 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:28.574721 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:36.491072 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:36.491026 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:36.559192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:36.559157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:40.670159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:40.670119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:40.761997 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:40.761966 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:45.114871 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:45.114828 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:45.218800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:45.218759 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:53.170460 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:53.170401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:53.270843 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:53.270810 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:58.322979 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:58.322947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:58.424475 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:58.424438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:58.976748 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:58.976711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:59.084780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.084741 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:59.155037 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.155006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:59.914452 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914416 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914722 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914735 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914743 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb2c9056-5205-457c-86cf-cccd6a8891e1" containerName="console" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914748 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb2c9056-5205-457c-86cf-cccd6a8891e1" containerName="console" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914758 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914764 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914771 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914776 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914784 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914789 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914795 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914801 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914809 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914814 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="util" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914820 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="extract" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914825 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="extract" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914830 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914835 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="pull" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914843 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="extract" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914848 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="extract" Jul 14 17:12:59.914841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914853 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914858 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914866 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="pull" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914871 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="pull" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914880 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914885 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914931 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="578685c7-a6d8-401d-9bc8-d45ebed7d344" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914938 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8bbfd0a-e37e-481b-879e-d8b898ad5599" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914945 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff7519d1-65c8-4936-b255-a4def886833c" containerName="extract" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914950 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb2c9056-5205-457c-86cf-cccd6a8891e1" containerName="console" Jul 14 17:12:59.915456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.914956 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d5e81c1-2550-4aaa-a881-e146827f07c0" containerName="extract" Jul 14 17:12:59.916668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.916649 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:12:59.920422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.920395 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 17:12:59.920711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.920698 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 17:12:59.924913 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.924897 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5dgdp\"" Jul 14 17:12:59.931526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.931500 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:12:59.953873 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.953838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:12:59.954286 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:12:59.954266 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27rcb\" (UniqueName: \"kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb\") pod \"authorino-58cd955675-xqgj7\" (UID: \"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169\") " pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:00.034790 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.034752 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:00.054957 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.054921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27rcb\" (UniqueName: \"kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb\") pod \"authorino-58cd955675-xqgj7\" (UID: \"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169\") " pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:00.065681 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.065647 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27rcb\" (UniqueName: \"kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb\") pod \"authorino-58cd955675-xqgj7\" (UID: \"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169\") " pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:00.163731 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.163671 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:00.165939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.165893 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:00.176864 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.176841 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:00.226425 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.226391 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:00.256756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.256725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmrzt\" (UniqueName: \"kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt\") pod \"authorino-75d556bd77-pzg52\" (UID: \"d001a392-1eba-40aa-a89c-3dc47853fb3c\") " pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:00.357314 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.357283 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:13:00.357641 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.357612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmrzt\" (UniqueName: \"kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt\") pod \"authorino-75d556bd77-pzg52\" (UID: \"d001a392-1eba-40aa-a89c-3dc47853fb3c\") " pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:00.359230 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:13:00.359201 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b3972fe_b4fd_4039_b0a6_c77dd8ac1169.slice/crio-7d99294966ee719a50297ecf8987e1c7994003e6bf626fc054e84c20227aa0d1 WatchSource:0}: Error finding container 7d99294966ee719a50297ecf8987e1c7994003e6bf626fc054e84c20227aa0d1: Status 404 returned error can't find the container with id 7d99294966ee719a50297ecf8987e1c7994003e6bf626fc054e84c20227aa0d1 Jul 14 17:13:00.367052 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.367030 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmrzt\" (UniqueName: \"kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt\") pod \"authorino-75d556bd77-pzg52\" (UID: \"d001a392-1eba-40aa-a89c-3dc47853fb3c\") " pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:00.474262 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.474146 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:00.608724 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.608697 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:00.609832 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:13:00.609810 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd001a392_1eba_40aa_a89c_3dc47853fb3c.slice/crio-1b3b3882bd0e13a41602afe335144d898a8d84460c6b63175e088f348d1cc015 WatchSource:0}: Error finding container 1b3b3882bd0e13a41602afe335144d898a8d84460c6b63175e088f348d1cc015: Status 404 returned error can't find the container with id 1b3b3882bd0e13a41602afe335144d898a8d84460c6b63175e088f348d1cc015 Jul 14 17:13:00.854745 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.854706 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzg52" event={"ID":"d001a392-1eba-40aa-a89c-3dc47853fb3c","Type":"ContainerStarted","Data":"1b3b3882bd0e13a41602afe335144d898a8d84460c6b63175e088f348d1cc015"} Jul 14 17:13:00.855770 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:00.855742 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xqgj7" event={"ID":"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169","Type":"ContainerStarted","Data":"7d99294966ee719a50297ecf8987e1c7994003e6bf626fc054e84c20227aa0d1"} Jul 14 17:13:04.880217 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:04.880178 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzg52" event={"ID":"d001a392-1eba-40aa-a89c-3dc47853fb3c","Type":"ContainerStarted","Data":"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d"} Jul 14 17:13:04.881808 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:04.881780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xqgj7" event={"ID":"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169","Type":"ContainerStarted","Data":"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d"} Jul 14 17:13:04.899358 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:04.899305 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-pzg52" podStartSLOduration=1.546397509 podStartE2EDuration="4.899289983s" totalImagesPullingTime="3.352892474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:04.896759198 +0000 UTC m=+478.419699983" watchObservedRunningTime="2026-07-14 17:13:04.899289983 +0000 UTC m=+478.422230734" Jul 14 17:13:04.937062 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:04.937003 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xqgj7" podStartSLOduration=2.344994966 podStartE2EDuration="5.936981273s" totalImagesPullingTime="3.591986307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:04.928273287 +0000 UTC m=+478.451214058" watchObservedRunningTime="2026-07-14 17:13:04.936981273 +0000 UTC m=+478.459922046" Jul 14 17:13:04.942098 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:04.942061 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:13:05.734100 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:05.734064 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:05.912141 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:05.912109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:06.890798 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:06.890757 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xqgj7" podUID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" containerName="authorino" containerID="cri-o://53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d" gracePeriod=30 Jul 14 17:13:07.133509 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.133476 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:07.211421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.211329 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27rcb\" (UniqueName: \"kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb\") pod \"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169\" (UID: \"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169\") " Jul 14 17:13:07.213431 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.213400 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb" (OuterVolumeSpecName: "kube-api-access-27rcb") pod "0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" (UID: "0b3972fe-b4fd-4039-b0a6-c77dd8ac1169"). InnerVolumeSpecName "kube-api-access-27rcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:13:07.312063 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.312020 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27rcb\" (UniqueName: \"kubernetes.io/projected/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169-kube-api-access-27rcb\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:07.896373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.896337 2574 generic.go:356] "Generic (PLEG): container finished" podID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" containerID="53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d" exitCode=0 Jul 14 17:13:07.896373 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.896373 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xqgj7" Jul 14 17:13:07.896615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.896416 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xqgj7" event={"ID":"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169","Type":"ContainerDied","Data":"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d"} Jul 14 17:13:07.896615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.896458 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xqgj7" event={"ID":"0b3972fe-b4fd-4039-b0a6-c77dd8ac1169","Type":"ContainerDied","Data":"7d99294966ee719a50297ecf8987e1c7994003e6bf626fc054e84c20227aa0d1"} Jul 14 17:13:07.896615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.896480 2574 scope.go:117] "RemoveContainer" containerID="53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d" Jul 14 17:13:07.904536 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.904516 2574 scope.go:117] "RemoveContainer" containerID="53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d" Jul 14 17:13:07.904829 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:07.904800 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d\": container with ID starting with 53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d not found: ID does not exist" containerID="53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d" Jul 14 17:13:07.904891 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.904831 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d"} err="failed to get container status \"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d\": rpc error: code = NotFound desc = could not find container \"53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d\": container with ID starting with 53ffcf7d349fc0fe6dae2882b8c9fd19b028fa1ed70efcfadd51a95c5dc1a45d not found: ID does not exist" Jul 14 17:13:07.915054 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.915031 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:13:07.921423 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:07.921396 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xqgj7"] Jul 14 17:13:09.691867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:09.691834 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" path="/var/lib/kubelet/pods/0b3972fe-b4fd-4039-b0a6-c77dd8ac1169/volumes" Jul 14 17:13:20.206388 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:20.206345 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:20.306666 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:20.306626 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:20.716280 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:20.716243 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:33.409178 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.409144 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:33.528715 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.528676 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:33.644082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.644047 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:33.644374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.644352 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" containerName="authorino" Jul 14 17:13:33.644374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.644368 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" containerName="authorino" Jul 14 17:13:33.644561 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.644422 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b3972fe-b4fd-4039-b0a6-c77dd8ac1169" containerName="authorino" Jul 14 17:13:33.650475 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.650454 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:33.653526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.653500 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:33.724422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.724330 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n98q\" (UniqueName: \"kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q\") pod \"authorino-5d4b67c446-6d85g\" (UID: \"1012df33-cd51-490d-b9db-c5f3efed1d00\") " pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:33.825752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.825708 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n98q\" (UniqueName: \"kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q\") pod \"authorino-5d4b67c446-6d85g\" (UID: \"1012df33-cd51-490d-b9db-c5f3efed1d00\") " pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:33.835018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.834994 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n98q\" (UniqueName: \"kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q\") pod \"authorino-5d4b67c446-6d85g\" (UID: \"1012df33-cd51-490d-b9db-c5f3efed1d00\") " pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:33.881208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.881174 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:33.881413 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.881402 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:33.913254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.913221 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76c599f457-vffc2"] Jul 14 17:13:33.917177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.917155 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:33.926594 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:33.926546 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76c599f457-vffc2"] Jul 14 17:13:34.019180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.019096 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:34.022088 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:13:34.022062 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1012df33_cd51_490d_b9db_c5f3efed1d00.slice/crio-f80aeb7f14895fabaf4ae17f70c27dd194ccfbd8e9b226fa6411c27b3a7676f1 WatchSource:0}: Error finding container f80aeb7f14895fabaf4ae17f70c27dd194ccfbd8e9b226fa6411c27b3a7676f1: Status 404 returned error can't find the container with id f80aeb7f14895fabaf4ae17f70c27dd194ccfbd8e9b226fa6411c27b3a7676f1 Jul 14 17:13:34.027271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.027218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh892\" (UniqueName: \"kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892\") pod \"authorino-76c599f457-vffc2\" (UID: \"d0a5b79a-58f1-4bf8-a960-3fe0473253f0\") " pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:34.093070 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.093031 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76c599f457-vffc2"] Jul 14 17:13:34.093285 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:34.093267 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-wh892], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76c599f457-vffc2" podUID="d0a5b79a-58f1-4bf8-a960-3fe0473253f0" Jul 14 17:13:34.124457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.124419 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:13:34.127804 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.127766 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh892\" (UniqueName: \"kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892\") pod \"authorino-76c599f457-vffc2\" (UID: \"d0a5b79a-58f1-4bf8-a960-3fe0473253f0\") " pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:34.128163 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.128145 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.130813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.130792 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 17:13:34.138519 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.138495 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:13:34.140521 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.140492 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh892\" (UniqueName: \"kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892\") pod \"authorino-76c599f457-vffc2\" (UID: \"d0a5b79a-58f1-4bf8-a960-3fe0473253f0\") " pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:34.229157 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.229114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn4tz\" (UniqueName: \"kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.229342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.229173 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.330272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.330241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4tz\" (UniqueName: \"kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.330431 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.330291 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.332711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.332684 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.339527 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.339505 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4tz\" (UniqueName: \"kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz\") pod \"authorino-5d77d7fcf7-fg7n7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.438679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.438640 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:13:34.572999 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:34.572973 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:13:34.575453 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:13:34.575422 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4189ec17_f2a2_4a2f_b13f_cd629b4903a7.slice/crio-8be73e2fa07cacd5c09cb7fa2e4510876bf59c985b34d3be2cf30d583971423b WatchSource:0}: Error finding container 8be73e2fa07cacd5c09cb7fa2e4510876bf59c985b34d3be2cf30d583971423b: Status 404 returned error can't find the container with id 8be73e2fa07cacd5c09cb7fa2e4510876bf59c985b34d3be2cf30d583971423b Jul 14 17:13:35.019250 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.019215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6d85g" event={"ID":"1012df33-cd51-490d-b9db-c5f3efed1d00","Type":"ContainerStarted","Data":"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d"} Jul 14 17:13:35.019411 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.019264 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6d85g" event={"ID":"1012df33-cd51-490d-b9db-c5f3efed1d00","Type":"ContainerStarted","Data":"f80aeb7f14895fabaf4ae17f70c27dd194ccfbd8e9b226fa6411c27b3a7676f1"} Jul 14 17:13:35.019411 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.019274 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-6d85g" podUID="1012df33-cd51-490d-b9db-c5f3efed1d00" containerName="authorino" containerID="cri-o://b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d" gracePeriod=30 Jul 14 17:13:35.020622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.020590 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" event={"ID":"4189ec17-f2a2-4a2f-b13f-cd629b4903a7","Type":"ContainerStarted","Data":"8be73e2fa07cacd5c09cb7fa2e4510876bf59c985b34d3be2cf30d583971423b"} Jul 14 17:13:35.020622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.020610 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:35.028387 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.028326 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:35.038211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.038164 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-6d85g" podStartSLOduration=1.371778836 podStartE2EDuration="2.038148069s" totalImagesPullingTime="666.369233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:35.036609221 +0000 UTC m=+508.559549987" watchObservedRunningTime="2026-07-14 17:13:35.038148069 +0000 UTC m=+508.561088842" Jul 14 17:13:35.136441 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.136403 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wh892\" (UniqueName: \"kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892\") pod \"d0a5b79a-58f1-4bf8-a960-3fe0473253f0\" (UID: \"d0a5b79a-58f1-4bf8-a960-3fe0473253f0\") " Jul 14 17:13:35.138354 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.138326 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892" (OuterVolumeSpecName: "kube-api-access-wh892") pod "d0a5b79a-58f1-4bf8-a960-3fe0473253f0" (UID: "d0a5b79a-58f1-4bf8-a960-3fe0473253f0"). InnerVolumeSpecName "kube-api-access-wh892". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:13:35.237105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.237015 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wh892\" (UniqueName: \"kubernetes.io/projected/d0a5b79a-58f1-4bf8-a960-3fe0473253f0-kube-api-access-wh892\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:35.259375 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.259350 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:35.337976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.337937 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n98q\" (UniqueName: \"kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q\") pod \"1012df33-cd51-490d-b9db-c5f3efed1d00\" (UID: \"1012df33-cd51-490d-b9db-c5f3efed1d00\") " Jul 14 17:13:35.340043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.340015 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q" (OuterVolumeSpecName: "kube-api-access-2n98q") pod "1012df33-cd51-490d-b9db-c5f3efed1d00" (UID: "1012df33-cd51-490d-b9db-c5f3efed1d00"). InnerVolumeSpecName "kube-api-access-2n98q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:13:35.439398 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:35.439364 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n98q\" (UniqueName: \"kubernetes.io/projected/1012df33-cd51-490d-b9db-c5f3efed1d00-kube-api-access-2n98q\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:36.027392 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.027304 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" event={"ID":"4189ec17-f2a2-4a2f-b13f-cd629b4903a7","Type":"ContainerStarted","Data":"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da"} Jul 14 17:13:36.028540 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028510 2574 generic.go:356] "Generic (PLEG): container finished" podID="1012df33-cd51-490d-b9db-c5f3efed1d00" containerID="b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d" exitCode=0 Jul 14 17:13:36.028655 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028545 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-6d85g" Jul 14 17:13:36.028655 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6d85g" event={"ID":"1012df33-cd51-490d-b9db-c5f3efed1d00","Type":"ContainerDied","Data":"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d"} Jul 14 17:13:36.028655 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028606 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-6d85g" event={"ID":"1012df33-cd51-490d-b9db-c5f3efed1d00","Type":"ContainerDied","Data":"f80aeb7f14895fabaf4ae17f70c27dd194ccfbd8e9b226fa6411c27b3a7676f1"} Jul 14 17:13:36.028655 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028627 2574 scope.go:117] "RemoveContainer" containerID="b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d" Jul 14 17:13:36.028863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.028728 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76c599f457-vffc2" Jul 14 17:13:36.037358 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.037338 2574 scope.go:117] "RemoveContainer" containerID="b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d" Jul 14 17:13:36.037651 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:36.037627 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d\": container with ID starting with b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d not found: ID does not exist" containerID="b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d" Jul 14 17:13:36.037754 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.037658 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d"} err="failed to get container status \"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d\": rpc error: code = NotFound desc = could not find container \"b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d\": container with ID starting with b85f494fba53d829b21941948ddaede1f5c0a5b512e76a8124997997d27fe37d not found: ID does not exist" Jul 14 17:13:36.064099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.064051 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" podStartSLOduration=1.687768289 podStartE2EDuration="2.064036043s" totalImagesPullingTime="376.267754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:36.058594724 +0000 UTC m=+509.581535492" watchObservedRunningTime="2026-07-14 17:13:36.064036043 +0000 UTC m=+509.586976815" Jul 14 17:13:36.085136 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.085096 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:36.095514 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.095482 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-6d85g"] Jul 14 17:13:36.111086 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.111060 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:36.111238 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.111217 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-pzg52" podUID="d001a392-1eba-40aa-a89c-3dc47853fb3c" containerName="authorino" containerID="cri-o://54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d" gracePeriod=30 Jul 14 17:13:36.157732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.157698 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76c599f457-vffc2"] Jul 14 17:13:36.160472 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.160446 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76c599f457-vffc2"] Jul 14 17:13:36.365766 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.365742 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:36.448657 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.448618 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmrzt\" (UniqueName: \"kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt\") pod \"d001a392-1eba-40aa-a89c-3dc47853fb3c\" (UID: \"d001a392-1eba-40aa-a89c-3dc47853fb3c\") " Jul 14 17:13:36.450793 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.450767 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt" (OuterVolumeSpecName: "kube-api-access-wmrzt") pod "d001a392-1eba-40aa-a89c-3dc47853fb3c" (UID: "d001a392-1eba-40aa-a89c-3dc47853fb3c"). InnerVolumeSpecName "kube-api-access-wmrzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:13:36.549632 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:36.549530 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmrzt\" (UniqueName: \"kubernetes.io/projected/d001a392-1eba-40aa-a89c-3dc47853fb3c-kube-api-access-wmrzt\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:37.034271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.034238 2574 generic.go:356] "Generic (PLEG): container finished" podID="d001a392-1eba-40aa-a89c-3dc47853fb3c" containerID="54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d" exitCode=0 Jul 14 17:13:37.034470 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.034277 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pzg52" Jul 14 17:13:37.034470 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.034328 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzg52" event={"ID":"d001a392-1eba-40aa-a89c-3dc47853fb3c","Type":"ContainerDied","Data":"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d"} Jul 14 17:13:37.034470 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.034372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pzg52" event={"ID":"d001a392-1eba-40aa-a89c-3dc47853fb3c","Type":"ContainerDied","Data":"1b3b3882bd0e13a41602afe335144d898a8d84460c6b63175e088f348d1cc015"} Jul 14 17:13:37.034470 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.034389 2574 scope.go:117] "RemoveContainer" containerID="54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d" Jul 14 17:13:37.043414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.043395 2574 scope.go:117] "RemoveContainer" containerID="54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d" Jul 14 17:13:37.043725 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:37.043705 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d\": container with ID starting with 54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d not found: ID does not exist" containerID="54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d" Jul 14 17:13:37.043789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.043733 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d"} err="failed to get container status \"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d\": rpc error: code = NotFound desc = could not find container \"54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d\": container with ID starting with 54caba4ccdd30ef570ed339db11f7ceaea31b32b33fc58e02f6a1d2c7500ae1d not found: ID does not exist" Jul 14 17:13:37.059097 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.059069 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:37.063883 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.063857 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pzg52"] Jul 14 17:13:37.692200 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.692165 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1012df33-cd51-490d-b9db-c5f3efed1d00" path="/var/lib/kubelet/pods/1012df33-cd51-490d-b9db-c5f3efed1d00/volumes" Jul 14 17:13:37.692611 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.692494 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d001a392-1eba-40aa-a89c-3dc47853fb3c" path="/var/lib/kubelet/pods/d001a392-1eba-40aa-a89c-3dc47853fb3c/volumes" Jul 14 17:13:37.692803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:37.692789 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a5b79a-58f1-4bf8-a960-3fe0473253f0" path="/var/lib/kubelet/pods/d0a5b79a-58f1-4bf8-a960-3fe0473253f0/volumes" Jul 14 17:13:40.865215 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.865174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:40.919929 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.919889 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:40.920324 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920309 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1012df33-cd51-490d-b9db-c5f3efed1d00" containerName="authorino" Jul 14 17:13:40.920365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920336 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1012df33-cd51-490d-b9db-c5f3efed1d00" containerName="authorino" Jul 14 17:13:40.920365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920361 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d001a392-1eba-40aa-a89c-3dc47853fb3c" containerName="authorino" Jul 14 17:13:40.920432 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920370 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d001a392-1eba-40aa-a89c-3dc47853fb3c" containerName="authorino" Jul 14 17:13:40.920463 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920442 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1012df33-cd51-490d-b9db-c5f3efed1d00" containerName="authorino" Jul 14 17:13:40.920463 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.920455 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d001a392-1eba-40aa-a89c-3dc47853fb3c" containerName="authorino" Jul 14 17:13:40.923637 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.923613 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:40.926468 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.926444 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 17:13:40.926760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.926737 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 17:13:40.927668 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.927647 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r25cv\"" Jul 14 17:13:40.936798 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.936770 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:40.985431 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.985387 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:40.985657 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:40.985446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs7zb\" (UniqueName: \"kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.004830 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.004796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:41.086428 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.086388 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.086428 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.086431 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs7zb\" (UniqueName: \"kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.086752 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:41.086560 2574 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 14 17:13:41.086752 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:41.086668 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert podName:d4aa2f19-f75e-48f5-87ea-712597259e39 nodeName:}" failed. No retries permitted until 2026-07-14 17:13:41.586645481 +0000 UTC m=+515.109586241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert") pod "maas-controller-79bc5b4c67-lr497" (UID: "d4aa2f19-f75e-48f5-87ea-712597259e39") : secret "maas-controller-webhook-cert" not found Jul 14 17:13:41.101866 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.101838 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs7zb\" (UniqueName: \"kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.106047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.106012 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:41.268461 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.268358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:41.590614 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.590551 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.593009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.592985 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") pod \"maas-controller-79bc5b4c67-lr497\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.836090 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.836063 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:41.984715 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:41.984656 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:41.987058 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:13:41.987019 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4aa2f19_f75e_48f5_87ea_712597259e39.slice/crio-07d5459825c71341441dc21f6a617b7c032bb120b45c31f15d743c4b38ad9749 WatchSource:0}: Error finding container 07d5459825c71341441dc21f6a617b7c032bb120b45c31f15d743c4b38ad9749: Status 404 returned error can't find the container with id 07d5459825c71341441dc21f6a617b7c032bb120b45c31f15d743c4b38ad9749 Jul 14 17:13:42.063992 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:42.063956 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bc5b4c67-lr497" event={"ID":"d4aa2f19-f75e-48f5-87ea-712597259e39","Type":"ContainerStarted","Data":"07d5459825c71341441dc21f6a617b7c032bb120b45c31f15d743c4b38ad9749"} Jul 14 17:13:43.771956 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:43.771929 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:43.925104 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:43.925068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:13:44.077569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:44.077538 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bc5b4c67-lr497" event={"ID":"d4aa2f19-f75e-48f5-87ea-712597259e39","Type":"ContainerStarted","Data":"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6"} Jul 14 17:13:44.077749 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:44.077650 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:44.131116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:44.130945 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-79bc5b4c67-lr497" podStartSLOduration=2.370549674 podStartE2EDuration="4.130925622s" totalImagesPullingTime="1.760375948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:13:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:13:44.129195333 +0000 UTC m=+517.652136106" watchObservedRunningTime="2026-07-14 17:13:44.130925622 +0000 UTC m=+517.653866396" Jul 14 17:13:55.084901 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.084864 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:55.180685 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.180649 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:55.180917 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.180856 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-79bc5b4c67-lr497" podUID="d4aa2f19-f75e-48f5-87ea-712597259e39" containerName="manager" containerID="cri-o://ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6" gracePeriod=30 Jul 14 17:13:55.428179 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.428154 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:55.611202 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.611112 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs7zb\" (UniqueName: \"kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb\") pod \"d4aa2f19-f75e-48f5-87ea-712597259e39\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " Jul 14 17:13:55.611202 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.611166 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") pod \"d4aa2f19-f75e-48f5-87ea-712597259e39\" (UID: \"d4aa2f19-f75e-48f5-87ea-712597259e39\") " Jul 14 17:13:55.613334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.613300 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d4aa2f19-f75e-48f5-87ea-712597259e39" (UID: "d4aa2f19-f75e-48f5-87ea-712597259e39"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:13:55.613445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.613376 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb" (OuterVolumeSpecName: "kube-api-access-gs7zb") pod "d4aa2f19-f75e-48f5-87ea-712597259e39" (UID: "d4aa2f19-f75e-48f5-87ea-712597259e39"). InnerVolumeSpecName "kube-api-access-gs7zb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:13:55.712526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.712479 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs7zb\" (UniqueName: \"kubernetes.io/projected/d4aa2f19-f75e-48f5-87ea-712597259e39-kube-api-access-gs7zb\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:55.712526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:55.712521 2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d4aa2f19-f75e-48f5-87ea-712597259e39-webhook-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:13:56.136066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.136031 2574 generic.go:356] "Generic (PLEG): container finished" podID="d4aa2f19-f75e-48f5-87ea-712597259e39" containerID="ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6" exitCode=0 Jul 14 17:13:56.136491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.136071 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79bc5b4c67-lr497" Jul 14 17:13:56.136491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.136115 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bc5b4c67-lr497" event={"ID":"d4aa2f19-f75e-48f5-87ea-712597259e39","Type":"ContainerDied","Data":"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6"} Jul 14 17:13:56.136491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.136162 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79bc5b4c67-lr497" event={"ID":"d4aa2f19-f75e-48f5-87ea-712597259e39","Type":"ContainerDied","Data":"07d5459825c71341441dc21f6a617b7c032bb120b45c31f15d743c4b38ad9749"} Jul 14 17:13:56.136491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.136188 2574 scope.go:117] "RemoveContainer" containerID="ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6" Jul 14 17:13:56.144654 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.144631 2574 scope.go:117] "RemoveContainer" containerID="ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6" Jul 14 17:13:56.144958 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:13:56.144936 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6\": container with ID starting with ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6 not found: ID does not exist" containerID="ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6" Jul 14 17:13:56.145006 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.144970 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6"} err="failed to get container status \"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6\": rpc error: code = NotFound desc = could not find container \"ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6\": container with ID starting with ea9271a82a61e7656b1dfd9cb73a0d536dbd92b7b30e3ffd80bfb067a3d9a7c6 not found: ID does not exist" Jul 14 17:13:56.164562 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.164525 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:56.177317 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:56.177285 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79bc5b4c67-lr497"] Jul 14 17:13:57.692072 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:13:57.692039 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4aa2f19-f75e-48f5-87ea-712597259e39" path="/var/lib/kubelet/pods/d4aa2f19-f75e-48f5-87ea-712597259e39/volumes" Jul 14 17:14:15.351044 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:15.350962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:15.448457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:15.448407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:15.913395 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:15.913365 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:16.249940 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:16.249838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:16.426976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:16.426942 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:16.525212 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:16.525097 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:26.739743 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:26.739688 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:26.840417 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:26.840377 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:43.192445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:43.192407 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:46.750426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.748373 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7"] Jul 14 17:14:46.750426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.749023 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aa2f19-f75e-48f5-87ea-712597259e39" containerName="manager" Jul 14 17:14:46.750426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.749044 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aa2f19-f75e-48f5-87ea-712597259e39" containerName="manager" Jul 14 17:14:46.750426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.749175 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aa2f19-f75e-48f5-87ea-712597259e39" containerName="manager" Jul 14 17:14:46.753066 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.753047 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.757362 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.757333 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 17:14:46.757537 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.757360 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xt9hh\"" Jul 14 17:14:46.757537 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.757413 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 17:14:46.757687 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.757620 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 17:14:46.760387 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.760364 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7"] Jul 14 17:14:46.835193 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.835159 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:46.889281 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889239 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9pwg\" (UniqueName: \"kubernetes.io/projected/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-kube-api-access-l9pwg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.889445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.889445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.889445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.889445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889363 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.889445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.889384 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990657 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9pwg\" (UniqueName: \"kubernetes.io/projected/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-kube-api-access-l9pwg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990713 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990743 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990774 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990803 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.990916 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.990835 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.991213 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.991187 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.991293 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.991216 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.991293 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.991257 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.993099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.993076 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:46.993355 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:46.993333 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:47.001665 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:47.001610 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9pwg\" (UniqueName: \"kubernetes.io/projected/26c3f1f9-5c97-4278-ae0c-6df0611f89c0-kube-api-access-l9pwg\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qqqt7\" (UID: \"26c3f1f9-5c97-4278-ae0c-6df0611f89c0\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:47.063405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:47.063359 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:47.206765 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:47.206742 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7"] Jul 14 17:14:47.208682 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:14:47.208655 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c3f1f9_5c97_4278_ae0c_6df0611f89c0.slice/crio-5771ae6c922676e38b54adbcbdcd6629efd424fd1d42230d1ed97da1e19c1106 WatchSource:0}: Error finding container 5771ae6c922676e38b54adbcbdcd6629efd424fd1d42230d1ed97da1e19c1106: Status 404 returned error can't find the container with id 5771ae6c922676e38b54adbcbdcd6629efd424fd1d42230d1ed97da1e19c1106 Jul 14 17:14:47.374547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:47.374509 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" event={"ID":"26c3f1f9-5c97-4278-ae0c-6df0611f89c0","Type":"ContainerStarted","Data":"5771ae6c922676e38b54adbcbdcd6629efd424fd1d42230d1ed97da1e19c1106"} Jul 14 17:14:48.382722 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:48.382684 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" event={"ID":"26c3f1f9-5c97-4278-ae0c-6df0611f89c0","Type":"ContainerStarted","Data":"785bdd45d7c653145a35fa86c731c505b29c4e0301429ccb0fd736bc6c4708b6"} Jul 14 17:14:48.419476 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:48.419417 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" podStartSLOduration=1.3378213589999999 podStartE2EDuration="2.419398021s" totalImagesPullingTime="1.081576662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:14:48.417415346 +0000 UTC m=+581.940356120" watchObservedRunningTime="2026-07-14 17:14:48.419398021 +0000 UTC m=+581.942338793" Jul 14 17:14:53.775555 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:53.775518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:55.254342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:55.254307 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:14:57.063777 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:57.063739 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:57.064207 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:57.063807 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:57.076796 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:57.076762 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:14:57.439070 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:14:57.439039 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qqqt7" Jul 14 17:15:07.568532 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:07.568501 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:15:07.569042 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:07.568502 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:15:28.049828 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.049750 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv"] Jul 14 17:15:28.057663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.057639 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.063526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.063503 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 14 17:15:28.069279 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.069255 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv"] Jul 14 17:15:28.072147 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.072104 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:15:28.146483 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146449 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.146483 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.146735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146515 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.146735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146545 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdhm\" (UniqueName: \"kubernetes.io/projected/19cbca68-e09b-4238-991b-cf74068239b6-kube-api-access-2tdhm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.146735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19cbca68-e09b-4238-991b-cf74068239b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.146735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.146718 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247590 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247634 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdhm\" (UniqueName: \"kubernetes.io/projected/19cbca68-e09b-4238-991b-cf74068239b6-kube-api-access-2tdhm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19cbca68-e09b-4238-991b-cf74068239b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247762 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.247802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.247786 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.248146 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.248118 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.248146 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.248141 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.248319 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.248170 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.249908 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.249884 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19cbca68-e09b-4238-991b-cf74068239b6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.250200 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.250178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19cbca68-e09b-4238-991b-cf74068239b6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.258775 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.258753 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdhm\" (UniqueName: \"kubernetes.io/projected/19cbca68-e09b-4238-991b-cf74068239b6-kube-api-access-2tdhm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv\" (UID: \"19cbca68-e09b-4238-991b-cf74068239b6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.368453 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.368417 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:28.511676 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.511648 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv"] Jul 14 17:15:28.513822 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:15:28.513782 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19cbca68_e09b_4238_991b_cf74068239b6.slice/crio-052c8bdc8a397d09cff3b27800de555cdf29e7c8e327c2f11da46423f1086779 WatchSource:0}: Error finding container 052c8bdc8a397d09cff3b27800de555cdf29e7c8e327c2f11da46423f1086779: Status 404 returned error can't find the container with id 052c8bdc8a397d09cff3b27800de555cdf29e7c8e327c2f11da46423f1086779 Jul 14 17:15:28.515114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.515096 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:15:28.571824 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:28.571787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" event={"ID":"19cbca68-e09b-4238-991b-cf74068239b6","Type":"ContainerStarted","Data":"052c8bdc8a397d09cff3b27800de555cdf29e7c8e327c2f11da46423f1086779"} Jul 14 17:15:29.577879 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:29.577842 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" event={"ID":"19cbca68-e09b-4238-991b-cf74068239b6","Type":"ContainerStarted","Data":"0849de9eaac44929449daa2917b55879634bbf52282c291e975da72e9ef6a4a6"} Jul 14 17:15:29.602547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:29.602490 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" podStartSLOduration=1.398292635 podStartE2EDuration="1.602474757s" totalImagesPullingTime="204.182122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:15:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:15:29.601527717 +0000 UTC m=+623.124468487" watchObservedRunningTime="2026-07-14 17:15:29.602474757 +0000 UTC m=+623.125415683" Jul 14 17:15:32.513490 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:32.513461 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:15:38.368759 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:38.368710 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:38.368759 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:38.368772 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:38.381442 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:38.381412 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:38.633940 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:38.633853 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv" Jul 14 17:15:44.172541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.172509 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7"] Jul 14 17:15:44.175825 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.175804 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.194219 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.194188 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 14 17:15:44.229844 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.229810 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7"] Jul 14 17:15:44.247995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.247962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:15:44.289057 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.289244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289080 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b217724-f3e8-4699-bae5-fa22a6880fbe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.289244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289104 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.289244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289120 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.289244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289142 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4b67\" (UniqueName: \"kubernetes.io/projected/0b217724-f3e8-4699-bae5-fa22a6880fbe-kube-api-access-w4b67\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.289244 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.289168 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390252 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390202 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390272 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b217724-f3e8-4699-bae5-fa22a6880fbe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4b67\" (UniqueName: \"kubernetes.io/projected/0b217724-f3e8-4699-bae5-fa22a6880fbe-kube-api-access-w4b67\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390753 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390702 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390865 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390834 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.390944 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.390842 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.392666 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.392646 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b217724-f3e8-4699-bae5-fa22a6880fbe-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.392822 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.392804 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b217724-f3e8-4699-bae5-fa22a6880fbe-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.414771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.414737 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4b67\" (UniqueName: \"kubernetes.io/projected/0b217724-f3e8-4699-bae5-fa22a6880fbe-kube-api-access-w4b67\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7\" (UID: \"0b217724-f3e8-4699-bae5-fa22a6880fbe\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.484820 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.484730 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:44.633227 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.631717 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7"] Jul 14 17:15:44.633726 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:15:44.633695 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b217724_f3e8_4699_bae5_fa22a6880fbe.slice/crio-2742124ec5882d623c6551b30237f6e02210c51c6b1fe06a4ea8fd728dd2893d WatchSource:0}: Error finding container 2742124ec5882d623c6551b30237f6e02210c51c6b1fe06a4ea8fd728dd2893d: Status 404 returned error can't find the container with id 2742124ec5882d623c6551b30237f6e02210c51c6b1fe06a4ea8fd728dd2893d Jul 14 17:15:44.654672 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:44.654638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" event={"ID":"0b217724-f3e8-4699-bae5-fa22a6880fbe","Type":"ContainerStarted","Data":"2742124ec5882d623c6551b30237f6e02210c51c6b1fe06a4ea8fd728dd2893d"} Jul 14 17:15:45.662001 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:45.661965 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" event={"ID":"0b217724-f3e8-4699-bae5-fa22a6880fbe","Type":"ContainerStarted","Data":"fe5f0935ef01100169f35e800ebb1d22bd09ab69578d1bfee0aa051adb1a2584"} Jul 14 17:15:45.687560 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:45.687499 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" podStartSLOduration=1.469505122 podStartE2EDuration="1.687482678s" totalImagesPullingTime="217.977556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:15:45.686267781 +0000 UTC m=+639.209208556" watchObservedRunningTime="2026-07-14 17:15:45.687482678 +0000 UTC m=+639.210423451" Jul 14 17:15:54.485552 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:54.485505 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:54.485979 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:54.485566 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:54.498237 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:54.498202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:15:54.717849 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:15:54.717819 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7" Jul 14 17:16:47.955531 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:16:47.955442 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:16:48.062253 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:16:48.062216 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:17:04.276269 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.276168 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c47c5648d-frff2"] Jul 14 17:17:04.279511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.279488 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.291863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.291835 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c47c5648d-frff2"] Jul 14 17:17:04.374815 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.374782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e096c0-2a88-4c81-839e-fc0447eddf6b-tls-cert\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.374986 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.374824 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfh2d\" (UniqueName: \"kubernetes.io/projected/58e096c0-2a88-4c81-839e-fc0447eddf6b-kube-api-access-kfh2d\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.475997 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.475948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfh2d\" (UniqueName: \"kubernetes.io/projected/58e096c0-2a88-4c81-839e-fc0447eddf6b-kube-api-access-kfh2d\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.476186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.476057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e096c0-2a88-4c81-839e-fc0447eddf6b-tls-cert\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.478406 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.478386 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/58e096c0-2a88-4c81-839e-fc0447eddf6b-tls-cert\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.485964 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.485943 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfh2d\" (UniqueName: \"kubernetes.io/projected/58e096c0-2a88-4c81-839e-fc0447eddf6b-kube-api-access-kfh2d\") pod \"authorino-6c47c5648d-frff2\" (UID: \"58e096c0-2a88-4c81-839e-fc0447eddf6b\") " pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.587656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.587626 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c47c5648d-frff2" Jul 14 17:17:04.697853 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.697804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:17:04.724840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.724818 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c47c5648d-frff2"] Jul 14 17:17:04.727209 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:17:04.727180 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58e096c0_2a88_4c81_839e_fc0447eddf6b.slice/crio-7560b0d69d84d48d5484289d5e0d179d3110f4ae1f56841f6f25c28ec5e0678d WatchSource:0}: Error finding container 7560b0d69d84d48d5484289d5e0d179d3110f4ae1f56841f6f25c28ec5e0678d: Status 404 returned error can't find the container with id 7560b0d69d84d48d5484289d5e0d179d3110f4ae1f56841f6f25c28ec5e0678d Jul 14 17:17:04.791303 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:04.791263 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:17:05.038349 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:05.038260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c47c5648d-frff2" event={"ID":"58e096c0-2a88-4c81-839e-fc0447eddf6b","Type":"ContainerStarted","Data":"7560b0d69d84d48d5484289d5e0d179d3110f4ae1f56841f6f25c28ec5e0678d"} Jul 14 17:17:05.882824 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:05.882785 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:17:05.975363 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:05.975307 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:17:06.047506 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.047468 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c47c5648d-frff2" event={"ID":"58e096c0-2a88-4c81-839e-fc0447eddf6b","Type":"ContainerStarted","Data":"f7bc022114c2613ee8332b964545e7c86c055d97dd41fb42749467c23296b500"} Jul 14 17:17:06.127953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.127888 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c47c5648d-frff2" podStartSLOduration=1.651916341 podStartE2EDuration="2.127868317s" totalImagesPullingTime="475.951976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:17:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:17:06.079416801 +0000 UTC m=+719.602357575" watchObservedRunningTime="2026-07-14 17:17:06.127868317 +0000 UTC m=+719.650809090" Jul 14 17:17:06.129348 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.129323 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:17:06.129566 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.129539 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" podUID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" containerName="authorino" containerID="cri-o://b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da" gracePeriod=30 Jul 14 17:17:06.390674 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.390651 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:17:06.492799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.492712 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert\") pod \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " Jul 14 17:17:06.492799 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.492779 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn4tz\" (UniqueName: \"kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz\") pod \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\" (UID: \"4189ec17-f2a2-4a2f-b13f-cd629b4903a7\") " Jul 14 17:17:06.494835 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.494798 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz" (OuterVolumeSpecName: "kube-api-access-fn4tz") pod "4189ec17-f2a2-4a2f-b13f-cd629b4903a7" (UID: "4189ec17-f2a2-4a2f-b13f-cd629b4903a7"). InnerVolumeSpecName "kube-api-access-fn4tz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:17:06.503898 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.503860 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4189ec17-f2a2-4a2f-b13f-cd629b4903a7" (UID: "4189ec17-f2a2-4a2f-b13f-cd629b4903a7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:17:06.594380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.594342 2574 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-tls-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:17:06.594380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:06.594373 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn4tz\" (UniqueName: \"kubernetes.io/projected/4189ec17-f2a2-4a2f-b13f-cd629b4903a7-kube-api-access-fn4tz\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:17:07.053633 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.053602 2574 generic.go:356] "Generic (PLEG): container finished" podID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" containerID="b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da" exitCode=0 Jul 14 17:17:07.054082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.053646 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" Jul 14 17:17:07.054082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.053688 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" event={"ID":"4189ec17-f2a2-4a2f-b13f-cd629b4903a7","Type":"ContainerDied","Data":"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da"} Jul 14 17:17:07.054082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.053734 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d77d7fcf7-fg7n7" event={"ID":"4189ec17-f2a2-4a2f-b13f-cd629b4903a7","Type":"ContainerDied","Data":"8be73e2fa07cacd5c09cb7fa2e4510876bf59c985b34d3be2cf30d583971423b"} Jul 14 17:17:07.054082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.053754 2574 scope.go:117] "RemoveContainer" containerID="b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da" Jul 14 17:17:07.064911 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.064856 2574 scope.go:117] "RemoveContainer" containerID="b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da" Jul 14 17:17:07.065391 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:17:07.065314 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da\": container with ID starting with b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da not found: ID does not exist" containerID="b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da" Jul 14 17:17:07.065391 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.065353 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da"} err="failed to get container status \"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da\": rpc error: code = NotFound desc = could not find container \"b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da\": container with ID starting with b005dfc207f7fa78f9294bd116b97868f7452b731714da231d0ca63943a8f4da not found: ID does not exist" Jul 14 17:17:07.089063 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.089030 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:17:07.092440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.092414 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d77d7fcf7-fg7n7"] Jul 14 17:17:07.693492 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:17:07.693457 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" path="/var/lib/kubelet/pods/4189ec17-f2a2-4a2f-b13f-cd629b4903a7/volumes" Jul 14 17:18:16.917961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:16.917919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:18.229458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.229412 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:18.359816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.359785 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-64846ff95c-nd78g"] Jul 14 17:18:18.360143 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.360130 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" containerName="authorino" Jul 14 17:18:18.360186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.360146 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" containerName="authorino" Jul 14 17:18:18.360221 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.360204 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4189ec17-f2a2-4a2f-b13f-cd629b4903a7" containerName="authorino" Jul 14 17:18:18.363146 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.363130 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.367432 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.367401 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 17:18:18.367588 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.367448 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 17:18:18.367588 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.367452 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-r25cv\"" Jul 14 17:18:18.374848 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.374825 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64846ff95c-nd78g"] Jul 14 17:18:18.419044 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.419002 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1e6b131-607c-4fe5-b265-22333a5c5204-webhook-cert\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.419229 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.419065 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p8kq\" (UniqueName: \"kubernetes.io/projected/a1e6b131-607c-4fe5-b265-22333a5c5204-kube-api-access-8p8kq\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.519693 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.519556 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p8kq\" (UniqueName: \"kubernetes.io/projected/a1e6b131-607c-4fe5-b265-22333a5c5204-kube-api-access-8p8kq\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.519693 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.519693 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1e6b131-607c-4fe5-b265-22333a5c5204-webhook-cert\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.522123 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.522096 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1e6b131-607c-4fe5-b265-22333a5c5204-webhook-cert\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.529884 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.529861 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p8kq\" (UniqueName: \"kubernetes.io/projected/a1e6b131-607c-4fe5-b265-22333a5c5204-kube-api-access-8p8kq\") pod \"maas-controller-64846ff95c-nd78g\" (UID: \"a1e6b131-607c-4fe5-b265-22333a5c5204\") " pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.675184 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.675147 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:18.809622 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:18.809597 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-64846ff95c-nd78g"] Jul 14 17:18:18.811461 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:18:18.811434 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1e6b131_607c_4fe5_b265_22333a5c5204.slice/crio-cd002b498fe1eea548eefa0bd56c9b86304756450cc326a6c08eb286959226df WatchSource:0}: Error finding container cd002b498fe1eea548eefa0bd56c9b86304756450cc326a6c08eb286959226df: Status 404 returned error can't find the container with id cd002b498fe1eea548eefa0bd56c9b86304756450cc326a6c08eb286959226df Jul 14 17:18:19.396172 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:19.396138 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-nd78g" event={"ID":"a1e6b131-607c-4fe5-b265-22333a5c5204","Type":"ContainerStarted","Data":"cd002b498fe1eea548eefa0bd56c9b86304756450cc326a6c08eb286959226df"} Jul 14 17:18:20.402899 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:20.402867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-64846ff95c-nd78g" event={"ID":"a1e6b131-607c-4fe5-b265-22333a5c5204","Type":"ContainerStarted","Data":"8c299d488d77f2d84684d5aa3588d225b7c7971b69653b795ab0d61e221a4e57"} Jul 14 17:18:20.403339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:20.402943 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:20.425734 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:20.425685 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-64846ff95c-nd78g" podStartSLOduration=1.781911434 podStartE2EDuration="2.425670577s" totalImagesPullingTime="643.759143ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:18:20.423496634 +0000 UTC m=+793.946437407" watchObservedRunningTime="2026-07-14 17:18:20.425670577 +0000 UTC m=+793.948611350" Jul 14 17:18:31.411079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:31.410998 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-64846ff95c-nd78g" Jul 14 17:18:46.159430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:46.159394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:53.144285 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:53.144244 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:57.915374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:57.915341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:58.012735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:58.012701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:18:58.173062 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:18:58.172963 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:04.770634 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:04.770598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:04.874354 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:04.874321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:04.971218 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:04.971179 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:05.150862 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:05.150817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:11.869076 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:11.869032 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:19:11.969137 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:19:11.969095 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:20:07.602800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:20:07.602725 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:20:07.603488 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:20:07.603001 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:20:08.947550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:20:08.947509 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:21:17.708547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:21:17.708505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:22:57.407221 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:22:57.407185 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:25:07.634223 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:25:07.634183 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:25:07.635372 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:25:07.635350 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:26:45.120156 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:26:45.120119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:27:29.559515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:27:29.559437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:28:21.987535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:28:21.987501 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:28:22.090119 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:28:22.090083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:07.669721 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:07.669694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:30:07.671681 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:07.671659 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:30:12.161043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:12.161013 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:18.291815 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:18.291778 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:26.652995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:26.652957 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:31.717786 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:31.717699 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:35.389034 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:35.388997 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:35.456195 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:35.456161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:41.657706 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.657665 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:41.744635 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.744597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:41.923839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.923758 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:30:41.927287 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.927264 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.931505 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.931482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fa58286c-openshift-default-dockercfg-tsmnq\"" Jul 14 17:30:41.935929 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.935906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:41.969380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.969348 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:30:41.982134 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982188 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmgxd\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982257 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982293 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982316 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982372 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982397 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:41.982569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:41.982419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.042836 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.042789 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:42.083174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083134 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmgxd\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083176 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083200 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083216 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083247 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083354 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083430 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083383 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083817 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083634 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083817 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083690 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083923 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083833 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.083923 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.083883 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.084017 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.084002 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.085895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.085870 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.086296 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.086277 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.098962 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.098934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.099331 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.099311 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmgxd\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd\") pod \"e2e-ait-fa58286c-openshift-default-5796989497-wmlb8\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.238455 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.238361 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:42.394390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.394361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:30:42.397000 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:30:42.396972 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c76bb37_eb92_4d59_8b09_1247fbcd77f4.slice/crio-d0d81de9e5558f6f78b1727a12314b914cff7ea1afb3b7efc0ad5fe780f75677 WatchSource:0}: Error finding container d0d81de9e5558f6f78b1727a12314b914cff7ea1afb3b7efc0ad5fe780f75677: Status 404 returned error can't find the container with id d0d81de9e5558f6f78b1727a12314b914cff7ea1afb3b7efc0ad5fe780f75677 Jul 14 17:30:42.399092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.399060 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:30:42.399189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.399131 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:30:42.399189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:42.399162 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:30:43.027602 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.027554 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerStarted","Data":"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4"} Jul 14 17:30:43.028102 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.027620 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerStarted","Data":"d0d81de9e5558f6f78b1727a12314b914cff7ea1afb3b7efc0ad5fe780f75677"} Jul 14 17:30:43.063480 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.063432 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podStartSLOduration=2.063413769 podStartE2EDuration="2.063413769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:30:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:30:43.063182662 +0000 UTC m=+1536.586123435" watchObservedRunningTime="2026-07-14 17:30:43.063413769 +0000 UTC m=+1536.586354541" Jul 14 17:30:43.092246 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.092213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:43.204217 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.204183 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:43.238858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.238829 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:43.243732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.243709 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:43.290966 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.290876 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:43.567211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.567117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:43.896076 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.896046 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:43.995041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:43.995006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:44.032317 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:44.032286 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:44.033376 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:44.033357 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:30:45.690290 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:45.690252 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:30:59.033332 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.033248 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:30:59.033731 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.033316 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:30:59.116128 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.116039 2574 generic.go:356] "Generic (PLEG): container finished" podID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerID="e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4" exitCode=0 Jul 14 17:30:59.116128 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.116112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerDied","Data":"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4"} Jul 14 17:30:59.116540 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.116525 2574 scope.go:117] "RemoveContainer" containerID="e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4" Jul 14 17:30:59.117708 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.117680 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:30:59.117780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.117763 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:30:59.117841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:30:59.117827 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:31:00.122951 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:00.122921 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerStarted","Data":"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a"} Jul 14 17:31:00.239391 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:00.239346 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:31:00.239674 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:00.239651 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:31:00.239779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:00.239706 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:31:01.238983 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:01.238949 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:31:01.239368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:01.239002 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:31:02.239670 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:02.239632 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:31:02.240067 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:02.239692 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:31:03.239789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:03.239749 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:31:03.240183 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:03.239812 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:31:04.239765 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:04.239719 2574 patch_prober.go:28] interesting pod/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 14 17:31:04.239950 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:04.239786 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 14 17:31:05.243334 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:05.243298 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:31:05.243838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:05.243609 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:31:05.244372 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:05.244354 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:31:58.744851 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:31:58.744772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:03.283438 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:03.283402 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:09.383616 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:09.383567 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:09.445695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:09.445660 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:13.462526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:13.462490 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:32:13.463001 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:13.462701 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" containerID="cri-o://505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a" gracePeriod=30 Jul 14 17:32:13.492617 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:13.492569 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:13.827780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:13.827750 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:13.927932 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:13.927902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:14.037111 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:14.037082 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:15.257083 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:15.257043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:15.381422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:15.381383 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:15.511470 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:15.511378 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:15.912260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:15.912223 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:16.026735 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:16.026700 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:18.704659 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.704631 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:32:18.821925 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.821894 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.821935 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.821964 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.821996 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822018 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822047 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822336 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822135 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmgxd\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822336 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822194 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:32:18.822336 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822215 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822336 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822262 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy\") pod \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\" (UID: \"1c76bb37-eb92-4d59-8b09-1247fbcd77f4\") " Jul 14 17:32:18.822541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822346 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:32:18.822541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822357 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:32:18.822541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822519 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:32:18.822731 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822532 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.822731 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822597 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.822731 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.822617 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.823097 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.823071 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data" (OuterVolumeSpecName: "istio-data") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:32:18.824297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.824267 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token" (OuterVolumeSpecName: "istio-token") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:32:18.824408 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.824311 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:32:18.824520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.824504 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:32:18.824635 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.824617 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd" (OuterVolumeSpecName: "kube-api-access-hmgxd") pod "1c76bb37-eb92-4d59-8b09-1247fbcd77f4" (UID: "1c76bb37-eb92-4d59-8b09-1247fbcd77f4"). InnerVolumeSpecName "kube-api-access-hmgxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:32:18.923599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923530 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.923599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923565 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.923599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923600 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.923599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923609 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.923869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923618 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmgxd\" (UniqueName: \"kubernetes.io/projected/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-kube-api-access-hmgxd\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:18.923869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:18.923628 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1c76bb37-eb92-4d59-8b09-1247fbcd77f4-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:32:19.508361 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.508329 2574 generic.go:356] "Generic (PLEG): container finished" podID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerID="505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a" exitCode=0 Jul 14 17:32:19.508535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.508374 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerDied","Data":"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a"} Jul 14 17:32:19.508535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.508379 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" Jul 14 17:32:19.508535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.508415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8" event={"ID":"1c76bb37-eb92-4d59-8b09-1247fbcd77f4","Type":"ContainerDied","Data":"d0d81de9e5558f6f78b1727a12314b914cff7ea1afb3b7efc0ad5fe780f75677"} Jul 14 17:32:19.508535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.508436 2574 scope.go:117] "RemoveContainer" containerID="505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a" Jul 14 17:32:19.517273 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.517256 2574 scope.go:117] "RemoveContainer" containerID="e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4" Jul 14 17:32:19.525252 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.525237 2574 scope.go:117] "RemoveContainer" containerID="505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a" Jul 14 17:32:19.525504 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:32:19.525487 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a\": container with ID starting with 505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a not found: ID does not exist" containerID="505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a" Jul 14 17:32:19.525556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.525513 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a"} err="failed to get container status \"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a\": rpc error: code = NotFound desc = could not find container \"505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a\": container with ID starting with 505415f774fd36af908a1f96568b97415fda396372d1e140d62b4dc04a76253a not found: ID does not exist" Jul 14 17:32:19.525556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.525529 2574 scope.go:117] "RemoveContainer" containerID="e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4" Jul 14 17:32:19.525773 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:32:19.525755 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4\": container with ID starting with e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4 not found: ID does not exist" containerID="e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4" Jul 14 17:32:19.525822 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.525779 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4"} err="failed to get container status \"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4\": rpc error: code = NotFound desc = could not find container \"e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4\": container with ID starting with e887af240902d716b3a680f4ce6182fb58ee8555659ff489934e00d5bf00f3c4 not found: ID does not exist" Jul 14 17:32:19.532991 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.532970 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:32:19.540870 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.540851 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fa58286c-openshift-default-5796989497-wmlb8"] Jul 14 17:32:19.692795 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:19.692762 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" path="/var/lib/kubelet/pods/1c76bb37-eb92-4d59-8b09-1247fbcd77f4/volumes" Jul 14 17:32:25.124113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:25.124079 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:25.189204 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:25.189161 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:32:28.362100 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:32:28.362063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:19.941887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:19.941828 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:20.040287 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:20.040250 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:25.904965 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:25.904924 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:26.339199 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:26.339168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:26.479532 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:26.479496 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:32.747701 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:32.747607 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:32.847186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:32.847149 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:33.545526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:33.545481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:33.646376 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:33.646339 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:34.234012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:34.233965 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:34.333398 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:34.333358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:34.875616 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:34.875547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:34.974250 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:34.974219 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:35.074316 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:35.074282 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:36.038682 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:36.038642 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:39.311422 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:39.311382 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:39.393270 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:39.393235 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:41.089356 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:41.089320 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:33:41.184390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:33:41.184359 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:34:03.013303 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:34:03.013271 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:34:29.253677 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:34:29.253645 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:34:29.328604 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:34:29.327039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:34:50.059805 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:34:50.059773 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:34:54.496729 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:34:54.496697 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:01.594442 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:01.594357 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:01.685593 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:01.685536 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:05.791210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:05.791172 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:06.057936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.057910 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:35:06.058243 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058232 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.058283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058245 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.058283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058257 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.058283 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058262 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.058374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058310 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.058374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.058319 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c76bb37-eb92-4d59-8b09-1247fbcd77f4" containerName="istio-proxy" Jul 14 17:35:06.061180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.061161 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.065538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.065516 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-74fcaf1a-openshift-default-dockercfg-x6p7j\"" Jul 14 17:35:06.079340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.079314 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:35:06.109977 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.109933 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:06.151977 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.151934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152053 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152117 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9cbl\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152224 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152241 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.152272 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.152258 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253419 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253375 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253419 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253414 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253435 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253566 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253633 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9cbl\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253659 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253675 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253694 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253990 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253729 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253990 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.253990 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.253931 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.254141 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.254071 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.254212 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.254190 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.254426 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.254404 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.256018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.255995 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.256103 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.256037 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.272748 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.272720 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9cbl\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.274739 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.274716 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token\") pod \"e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.371988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.371952 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:06.521685 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.521658 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:35:06.523726 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:35:06.523698 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd362c450_63d7_4d08_b78e_9efbefc18700.slice/crio-83ab01f5095fcc074922ce46d23eb2c7e593977826895d78a6b6bc71520c1543 WatchSource:0}: Error finding container 83ab01f5095fcc074922ce46d23eb2c7e593977826895d78a6b6bc71520c1543: Status 404 returned error can't find the container with id 83ab01f5095fcc074922ce46d23eb2c7e593977826895d78a6b6bc71520c1543 Jul 14 17:35:06.525887 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.525860 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:06.525945 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.525920 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:06.525982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:06.525952 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:07.301819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.301764 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerStarted","Data":"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c"} Jul 14 17:35:07.302274 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.301868 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerStarted","Data":"83ab01f5095fcc074922ce46d23eb2c7e593977826895d78a6b6bc71520c1543"} Jul 14 17:35:07.322630 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.322589 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:07.347533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.347477 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podStartSLOduration=1.347457655 podStartE2EDuration="1.347457655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:35:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:35:07.346078442 +0000 UTC m=+1800.869019214" watchObservedRunningTime="2026-07-14 17:35:07.347457655 +0000 UTC m=+1800.870398431" Jul 14 17:35:07.372696 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.372655 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:07.378776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.378746 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:07.444839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.444804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:07.657967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.657927 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:07.705778 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.705754 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:35:07.708126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:07.708101 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:35:08.307100 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:08.307065 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:08.308060 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:08.308039 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:08.388361 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:08.388327 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:08.480663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:08.480636 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:18.359018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.358983 2574 generic.go:356] "Generic (PLEG): container finished" podID="d362c450-63d7-4d08-b78e-9efbefc18700" containerID="d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c" exitCode=0 Jul 14 17:35:18.359441 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.359057 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerDied","Data":"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c"} Jul 14 17:35:18.359485 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.359462 2574 scope.go:117] "RemoveContainer" containerID="d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c" Jul 14 17:35:18.360423 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.360400 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:18.360521 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.360461 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:18.360521 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:18.360492 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:35:19.366756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.366724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerStarted","Data":"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170"} Jul 14 17:35:19.372082 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.372053 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:19.372345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.372324 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:19.373254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.373233 2574 patch_prober.go:28] interesting pod/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 17:35:19.373323 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.373284 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 17:35:19.477127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.477049 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:19.569777 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:19.569740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:20.372489 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:20.372457 2574 patch_prober.go:28] interesting pod/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 17:35:20.372905 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:20.372511 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 17:35:21.372433 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:21.372400 2574 patch_prober.go:28] interesting pod/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 17:35:21.372682 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:21.372455 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 17:35:22.335034 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:22.335000 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:22.373249 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:22.373203 2574 patch_prober.go:28] interesting pod/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 14 17:35:22.373675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:22.373269 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 14 17:35:23.307701 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:23.307658 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:23.376440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:23.376410 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:23.406897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:23.406870 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:35:55.325803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:55.325738 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:35:55.426671 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:35:55.426617 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:22.570817 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:22.570784 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:26.851676 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:26.851637 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:33.957938 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:33.957853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:34.028723 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:34.028685 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:37.108147 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:37.108111 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:36:37.108639 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:37.108322 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" containerID="cri-o://b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170" gracePeriod=30 Jul 14 17:36:37.141809 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:37.141775 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:37.827246 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:37.827210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:37.995199 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:37.995162 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:38.195073 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:38.194976 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:38.408159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:38.408113 2574 patch_prober.go:28] interesting pod/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 17:36:38.408345 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:38.408197 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:36:39.836755 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:39.836700 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:42.365145 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.365114 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:36:42.507240 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507205 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507260 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507299 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507324 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507445 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507373 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9cbl\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507508 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:36:42.507689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507586 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507633 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507671 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507703 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy\") pod \"d362c450-63d7-4d08-b78e-9efbefc18700\" (UID: \"d362c450-63d7-4d08-b78e-9efbefc18700\") " Jul 14 17:36:42.507958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507710 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:36:42.507958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507757 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:36:42.507958 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.507804 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:36:42.508170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.508021 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.508170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.508042 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.508170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.508055 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d362c450-63d7-4d08-b78e-9efbefc18700-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.508170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.508068 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.508170 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.508118 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data" (OuterVolumeSpecName: "istio-data") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:36:42.509772 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.509742 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token" (OuterVolumeSpecName: "istio-token") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:36:42.509884 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.509817 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl" (OuterVolumeSpecName: "kube-api-access-w9cbl") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "kube-api-access-w9cbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:36:42.510156 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.510134 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:36:42.510211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.510191 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d362c450-63d7-4d08-b78e-9efbefc18700" (UID: "d362c450-63d7-4d08-b78e-9efbefc18700"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:36:42.608502 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.608440 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.608502 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.608496 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9cbl\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-kube-api-access-w9cbl\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.608502 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.608511 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d362c450-63d7-4d08-b78e-9efbefc18700-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.608784 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.608522 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d362c450-63d7-4d08-b78e-9efbefc18700-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.608784 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.608533 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d362c450-63d7-4d08-b78e-9efbefc18700-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:36:42.795780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.795686 2574 generic.go:356] "Generic (PLEG): container finished" podID="d362c450-63d7-4d08-b78e-9efbefc18700" containerID="b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170" exitCode=0 Jul 14 17:36:42.795780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.795738 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" Jul 14 17:36:42.795780 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.795764 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerDied","Data":"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170"} Jul 14 17:36:42.796032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.795830 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8" event={"ID":"d362c450-63d7-4d08-b78e-9efbefc18700","Type":"ContainerDied","Data":"83ab01f5095fcc074922ce46d23eb2c7e593977826895d78a6b6bc71520c1543"} Jul 14 17:36:42.796032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.795855 2574 scope.go:117] "RemoveContainer" containerID="b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170" Jul 14 17:36:42.804996 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.804978 2574 scope.go:117] "RemoveContainer" containerID="d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c" Jul 14 17:36:42.814198 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.814172 2574 scope.go:117] "RemoveContainer" containerID="b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170" Jul 14 17:36:42.814541 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:36:42.814521 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170\": container with ID starting with b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170 not found: ID does not exist" containerID="b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170" Jul 14 17:36:42.814673 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.814565 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170"} err="failed to get container status \"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170\": rpc error: code = NotFound desc = could not find container \"b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170\": container with ID starting with b766ef7cc1e4f1d0a1357740febcb46345a8f19c495fbd781b330368f3b48170 not found: ID does not exist" Jul 14 17:36:42.814724 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.814677 2574 scope.go:117] "RemoveContainer" containerID="d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c" Jul 14 17:36:42.814948 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:36:42.814927 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c\": container with ID starting with d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c not found: ID does not exist" containerID="d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c" Jul 14 17:36:42.815041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.814951 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c"} err="failed to get container status \"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c\": rpc error: code = NotFound desc = could not find container \"d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c\": container with ID starting with d777d8e9c26ac4d5ba36ddde93477ef983eb963f4173c22a81cafee316bf143c not found: ID does not exist" Jul 14 17:36:42.856657 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.856622 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:36:42.860861 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:42.860836 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-74fcaf1a-openshift-default-647dd9d89f-qnmc8"] Jul 14 17:36:43.692909 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:43.692876 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" path="/var/lib/kubelet/pods/d362c450-63d7-4d08-b78e-9efbefc18700/volumes" Jul 14 17:36:44.366552 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:44.366515 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:49.208417 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:49.208385 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:49.462878 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:49.462790 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:49.717257 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:49.717168 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:36:51.142488 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:36:51.142454 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:17.652828 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:17.652794 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:23.678143 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:23.678108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:23.793840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:23.793805 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:23.896165 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:23.896129 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:39.885130 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:39.885090 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:46.660257 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:46.660217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:46.776119 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:46.776083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:46.920049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:46.919957 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:57.492367 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:57.492330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:37:58.362159 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:37:58.362068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:05.322992 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:05.322954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:05.400484 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:05.400445 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:05.525224 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:05.525187 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:05.634402 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:05.634367 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:06.484157 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:06.484128 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:06.634095 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:06.634065 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:06.756976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:06.756872 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:08.346424 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:08.346394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:15.043394 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:15.043361 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:15.148156 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:15.148122 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:15.745333 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:15.745296 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:22.204519 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:22.204483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:23.420065 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:23.420025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:23.521834 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:23.521800 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:25.530065 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:25.530023 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:25.648116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:25.648078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:25.733873 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:25.733839 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:26.014990 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:26.014953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:26.686332 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:26.686286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:26.785267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:26.785237 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:30.046966 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:30.046930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:30.779894 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:30.779865 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:30.917541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:30.917506 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:31.216041 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:31.216013 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:37.937865 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:37.937830 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:38.040449 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:38.040401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:40.202719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:40.202684 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:40.362774 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:40.362737 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:41.293266 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:41.293222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:41.375071 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:41.375040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:49.015487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:49.015449 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:38:53.489354 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:38:53.489318 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:01.134201 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:01.134165 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:01.239104 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:01.239070 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:04.319060 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:04.319026 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:04.529811 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:04.529771 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:04.694773 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:04.694742 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:04.798196 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:04.798163 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:04.939602 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:04.939541 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:11.839771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:11.839733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:11.946003 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:11.945968 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:12.046126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:12.046088 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:12.235862 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:12.235770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:20.879177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:20.879135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:20.982006 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:20.981959 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:27.965046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:27.964964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:28.330675 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.330640 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:39:28.330950 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.330937 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.331013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.330951 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.331013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.330959 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.331013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.330964 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.331167 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.331027 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.331167 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.331035 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d362c450-63d7-4d08-b78e-9efbefc18700" containerName="istio-proxy" Jul 14 17:39:28.333975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.333956 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.341800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.341782 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9cc7492b-openshift-default-dockercfg-mrpkk\"" Jul 14 17:39:28.362599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.362556 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:39:28.385167 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.385138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:28.414718 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414688 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414734 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414753 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvfn\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414804 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.414886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414866 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.415106 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414924 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.415106 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.414945 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516150 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516112 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516150 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516198 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516276 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvfn\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516302 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516342 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516365 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516628 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516655 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516929 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516906 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.516992 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516924 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.517046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.516987 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.518791 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.518772 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.518853 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.518770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.543632 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.543594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.544459 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.544437 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvfn\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn\") pod \"e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.603625 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.603507 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:28.643975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.643937 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:28.804720 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.804687 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:39:28.805031 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:39:28.805004 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9fac0a_3d3d_4242_bdeb_15c9d0128ebe.slice/crio-74c1a1e40c74310a98f2eff63451a214da4c86242537f6f7ad3a0c1379268904 WatchSource:0}: Error finding container 74c1a1e40c74310a98f2eff63451a214da4c86242537f6f7ad3a0c1379268904: Status 404 returned error can't find the container with id 74c1a1e40c74310a98f2eff63451a214da4c86242537f6f7ad3a0c1379268904 Jul 14 17:39:28.807113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.807081 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:28.807196 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.807147 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:28.807196 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:28.807177 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:29.602142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:29.602100 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerStarted","Data":"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55"} Jul 14 17:39:29.602547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:29.602184 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerStarted","Data":"74c1a1e40c74310a98f2eff63451a214da4c86242537f6f7ad3a0c1379268904"} Jul 14 17:39:29.644923 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:29.644881 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:29.650367 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:29.650339 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:29.676772 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:29.676720 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podStartSLOduration=1.6767047609999999 podStartE2EDuration="1.676704761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:39:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:39:29.674561202 +0000 UTC m=+2063.197501996" watchObservedRunningTime="2026-07-14 17:39:29.676704761 +0000 UTC m=+2063.199645535" Jul 14 17:39:30.607207 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:30.607175 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:30.608199 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:30.608180 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:31.874138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:31.874103 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:31.980813 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:31.980781 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:32.113050 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:32.113011 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:32.576760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:32.576728 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:32.808367 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:32.808329 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:32.963489 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:32.963391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:36.397071 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:36.397039 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:46.041684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.041125 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:36566->10.133.0.48:15021: read: connection reset by peer" start-of-body= Jul 14 17:39:46.041684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.041193 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:36566->10.133.0.48:15021: read: connection reset by peer" Jul 14 17:39:46.688993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.688957 2574 generic.go:356] "Generic (PLEG): container finished" podID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerID="a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55" exitCode=0 Jul 14 17:39:46.689177 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.689026 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerDied","Data":"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55"} Jul 14 17:39:46.689438 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.689425 2574 scope.go:117] "RemoveContainer" containerID="a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55" Jul 14 17:39:46.690406 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.690379 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:46.690499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.690482 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:46.690545 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:46.690531 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:39:47.695567 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:47.695535 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerStarted","Data":"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8"} Jul 14 17:39:48.644380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:48.644348 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:48.644597 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:48.644488 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 17:39:48.644597 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:48.644529 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 17:39:49.645262 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:49.645223 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 17:39:49.645703 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:49.645285 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 17:39:50.135427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:50.135394 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:39:50.644837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:50.644796 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 17:39:50.645026 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:50.644855 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 17:39:51.644595 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:51.644544 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 14 17:39:51.645022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:51.644628 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 14 17:39:52.648406 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:52.648372 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:52.648827 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:52.648690 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:39:52.649405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:39:52.649383 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:40:07.741303 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:07.741267 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:40:07.744565 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:07.744538 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:40:12.937161 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:12.937119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:40:21.275267 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:21.275230 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:40:23.398803 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:23.398770 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:40:25.357790 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:25.357752 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:40:30.421821 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:40:30.421772 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:07.100969 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:07.100892 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:08.315293 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:08.315256 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:13.840715 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:13.840674 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:14.061101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:14.061068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:17.662148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:17.662105 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:41:17.662677 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:17.662318 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" containerID="cri-o://b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8" gracePeriod=30 Jul 14 17:41:17.676949 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:17.676914 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:17.956263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:17.956174 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:19.011817 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:19.011784 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:19.108972 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:19.108938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:19.211359 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:19.211326 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:19.954695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:19.954658 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:20.052185 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:20.052150 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:20.160210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:20.160177 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:21.578315 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:21.578280 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:22.651589 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:22.651541 2574 patch_prober.go:28] interesting pod/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 17:41:22.651967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:22.651630 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:41:22.924396 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:22.924370 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:41:23.032586 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032549 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.032771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032616 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsvfn\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.032771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032675 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.032771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032719 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.032771 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032748 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.033008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032788 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.033008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032859 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.033008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032888 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.033008 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.032912 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs\") pod \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\" (UID: \"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe\") " Jul 14 17:41:23.033365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.033208 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:41:23.033365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.033307 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:41:23.033543 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.033363 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:41:23.033862 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.033833 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data" (OuterVolumeSpecName: "istio-data") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:41:23.033965 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.033904 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:41:23.034953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.034928 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:41:23.035160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.035136 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token" (OuterVolumeSpecName: "istio-token") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:41:23.035160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.035146 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn" (OuterVolumeSpecName: "kube-api-access-zsvfn") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "kube-api-access-zsvfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:41:23.035548 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.035533 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" (UID: "de9fac0a-3d3d-4242-bdeb-15c9d0128ebe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:41:23.133837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133798 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.133837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133833 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsvfn\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-kube-api-access-zsvfn\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.133837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133847 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133859 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133871 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133883 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133893 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133905 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.134092 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.133916 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:41:23.181263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.181166 2574 generic.go:356] "Generic (PLEG): container finished" podID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerID="b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8" exitCode=0 Jul 14 17:41:23.181263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.181217 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" Jul 14 17:41:23.181499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.181251 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerDied","Data":"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8"} Jul 14 17:41:23.181499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.181319 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9" event={"ID":"de9fac0a-3d3d-4242-bdeb-15c9d0128ebe","Type":"ContainerDied","Data":"74c1a1e40c74310a98f2eff63451a214da4c86242537f6f7ad3a0c1379268904"} Jul 14 17:41:23.181499 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.181351 2574 scope.go:117] "RemoveContainer" containerID="b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8" Jul 14 17:41:23.191128 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.191107 2574 scope.go:117] "RemoveContainer" containerID="a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55" Jul 14 17:41:23.200946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.200922 2574 scope.go:117] "RemoveContainer" containerID="b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8" Jul 14 17:41:23.201213 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:41:23.201192 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8\": container with ID starting with b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8 not found: ID does not exist" containerID="b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8" Jul 14 17:41:23.201271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.201222 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8"} err="failed to get container status \"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8\": rpc error: code = NotFound desc = could not find container \"b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8\": container with ID starting with b9150d0f58bc7f75a100787c7d4a727b271489e3ca887e36cc605a9e2df8e8c8 not found: ID does not exist" Jul 14 17:41:23.201271 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.201242 2574 scope.go:117] "RemoveContainer" containerID="a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55" Jul 14 17:41:23.201495 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:41:23.201482 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55\": container with ID starting with a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55 not found: ID does not exist" containerID="a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55" Jul 14 17:41:23.201542 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.201497 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55"} err="failed to get container status \"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55\": rpc error: code = NotFound desc = could not find container \"a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55\": container with ID starting with a778af185fbd7a4975fb63d90334038dd87dc525533dcc1a327d547f04130d55 not found: ID does not exist" Jul 14 17:41:23.209305 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.209278 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:41:23.213456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.213429 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9cc7492b-openshift-default-6b5c66758c-5bzp9"] Jul 14 17:41:23.693116 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:23.693081 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" path="/var/lib/kubelet/pods/de9fac0a-3d3d-4242-bdeb-15c9d0128ebe/volumes" Jul 14 17:41:27.905058 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:27.905019 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:28.006052 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:28.006021 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:28.584628 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:28.584587 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:35.451211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:35.451175 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:35.620965 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:35.620930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:35.793672 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:35.793554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:40.531701 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:40.531663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:40.645142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:40.645105 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:46.207043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:46.207007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:41:46.591329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:41:46.591284 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:00.206919 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:00.206883 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:01.634455 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:01.634418 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:07.607148 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:07.607097 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:14.084972 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.084935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:14.188169 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.188134 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:14.494081 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.493981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:14.789954 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.789878 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:42:14.790218 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790205 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.790263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790222 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.790263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790232 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.790263 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790238 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.790361 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790293 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.790361 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.790306 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de9fac0a-3d3d-4242-bdeb-15c9d0128ebe" containerName="istio-proxy" Jul 14 17:42:14.793171 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.793154 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.796863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.796846 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d44b45-openshift-default-dockercfg-cjvcs\"" Jul 14 17:42:14.808294 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.808250 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:42:14.808435 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.808335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:14.886012 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.885966 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886026 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886164 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886210 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886183 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqrsm\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886418 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886228 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886418 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886418 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886305 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.886418 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.886326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987490 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987431 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987490 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987538 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987614 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqrsm\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.987806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.987752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.988172 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.988037 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.988172 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.988130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.988443 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.988419 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.988760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.988733 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.988889 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.988747 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.990105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.990076 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.990507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.990484 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.999022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.998995 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:14.999160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:14.999123 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqrsm\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm\") pod \"e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:15.103015 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.102985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:15.110491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.110462 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:15.260149 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.260126 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:42:15.262293 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:42:15.262264 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdb40c9_7d56_4c8e_8076_ab27769f50f7.slice/crio-486cd09ac8d1c547d00dbd96f62bf451c00302a60efb79f4fc71ecef1964ef56 WatchSource:0}: Error finding container 486cd09ac8d1c547d00dbd96f62bf451c00302a60efb79f4fc71ecef1964ef56: Status 404 returned error can't find the container with id 486cd09ac8d1c547d00dbd96f62bf451c00302a60efb79f4fc71ecef1964ef56 Jul 14 17:42:15.264331 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.264291 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:15.264409 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.264365 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:15.264409 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.264393 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:15.438427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.438325 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerStarted","Data":"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730"} Jul 14 17:42:15.438427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.438379 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerStarted","Data":"486cd09ac8d1c547d00dbd96f62bf451c00302a60efb79f4fc71ecef1964ef56"} Jul 14 17:42:15.476910 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:15.476851 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podStartSLOduration=1.476835439 podStartE2EDuration="1.476835439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:15.475221073 +0000 UTC m=+2228.998161846" watchObservedRunningTime="2026-07-14 17:42:15.476835439 +0000 UTC m=+2228.999776212" Jul 14 17:42:16.103640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:16.103601 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:16.108700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:16.108677 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:16.444432 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:16.444346 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:16.445436 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:16.445419 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:17.562138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:17.562101 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:17.680757 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:17.680725 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:17.845045 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:17.845007 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:18.086433 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:18.086387 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:18.431605 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:18.431547 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:18.532550 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:18.532511 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:22.020076 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:22.020035 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:23.270794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:23.270761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:25.529363 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:25.529321 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:25.635142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:25.635109 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:25.904534 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:25.904499 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:26.173051 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:26.172964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:26.635619 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:26.635553 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:26.736685 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:26.736650 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:29.971091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:29.971053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:30.526999 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.526964 2574 generic.go:356] "Generic (PLEG): container finished" podID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerID="78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730" exitCode=0 Jul 14 17:42:30.527187 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.527025 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerDied","Data":"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730"} Jul 14 17:42:30.527396 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.527383 2574 scope.go:117] "RemoveContainer" containerID="78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730" Jul 14 17:42:30.528372 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.528342 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:30.528465 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.528424 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:30.528528 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:30.528486 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:42:31.444819 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:31.444784 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:31.533547 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:31.533508 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerStarted","Data":"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866"} Jul 14 17:42:32.103661 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:32.103621 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:32.104738 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:32.104716 2574 patch_prober.go:28] interesting pod/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 17:42:32.104801 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:32.104781 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 17:42:33.103939 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:33.103907 2574 patch_prober.go:28] interesting pod/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 17:42:33.104365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:33.103960 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 17:42:34.104532 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:34.104478 2574 patch_prober.go:28] interesting pod/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 17:42:34.104963 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:34.104558 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 17:42:35.103801 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:35.103763 2574 patch_prober.go:28] interesting pod/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body= Jul 14 17:42:35.103971 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:35.103820 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" Jul 14 17:42:36.107289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:36.107259 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:36.107711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:36.107692 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:36.108280 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:36.108265 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:42:39.264355 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:39.264303 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:39.360852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:39.360820 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:39.470831 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:39.470798 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:47.777967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:47.777929 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:47.884400 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:47.884350 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:53.339350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.339317 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:42:53.342860 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.342837 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.346487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.346464 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1b06a1\"/\"kube-root-ca.crt\"" Jul 14 17:42:53.346487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.346485 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1b06a1\"/\"openshift-service-ca.crt\"" Jul 14 17:42:53.346713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.346464 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1b06a1\"/\"test-model-1b06a1-kserve-self-signed-certs\"" Jul 14 17:42:53.346713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.346464 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1b06a1\"/\"default-dockercfg-gw5tg\"" Jul 14 17:42:53.353190 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.353167 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:42:53.372471 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.372429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:42:53.440789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.440751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.440789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.440789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.441062 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.440899 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.441062 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.440984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c42gx\" (UniqueName: \"kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.441062 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.441036 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.441212 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.441084 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.541776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.541736 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c42gx\" (UniqueName: \"kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.541955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.541800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.541955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.541923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.541969 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.541989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.542046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.542239 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542323 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.542294 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.542385 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.542359 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.544130 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.544106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.544382 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.544365 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.551235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.551214 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c42gx\" (UniqueName: \"kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx\") pod \"test-model-1b06a1-kserve-8d9d99fc5-z857f\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.653165 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.653071 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:42:53.789002 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:53.788972 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:42:53.791010 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:42:53.790979 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcab73c91_7f57_4668_adcb_95940723a0b6.slice/crio-5ed7f1f0461f6e44d6557451695b9de290fbab2814ae8bdddb10467d68c60358 WatchSource:0}: Error finding container 5ed7f1f0461f6e44d6557451695b9de290fbab2814ae8bdddb10467d68c60358: Status 404 returned error can't find the container with id 5ed7f1f0461f6e44d6557451695b9de290fbab2814ae8bdddb10467d68c60358 Jul 14 17:42:54.648852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:54.648817 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" event={"ID":"cab73c91-7f57-4668-adcb-95940723a0b6","Type":"ContainerStarted","Data":"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1"} Jul 14 17:42:54.648852 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:54.648863 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" event={"ID":"cab73c91-7f57-4668-adcb-95940723a0b6","Type":"ContainerStarted","Data":"5ed7f1f0461f6e44d6557451695b9de290fbab2814ae8bdddb10467d68c60358"} Jul 14 17:42:54.672007 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:42:54.671956 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" podStartSLOduration=1.67193882 podStartE2EDuration="1.67193882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:42:54.669793317 +0000 UTC m=+2268.192734091" watchObservedRunningTime="2026-07-14 17:42:54.67193882 +0000 UTC m=+2268.194879595" Jul 14 17:43:03.653589 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:03.653535 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:03.654005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:03.653630 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:03.666520 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:03.666494 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:03.703919 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:03.703891 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:13.190765 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:13.190716 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:14.092218 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:14.092182 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:43:14.105013 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:14.104977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:14.855311 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:14.855273 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:14.951754 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:14.951719 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:24.799621 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:24.799565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:29.112545 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.112508 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" podUID="cab73c91-7f57-4668-adcb-95940723a0b6" containerName="main" containerID="cri-o://e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1" gracePeriod=45 Jul 14 17:43:29.336516 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.336476 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:29.372157 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.372095 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:29.443195 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443163 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443399 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443236 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443399 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443304 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443399 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443378 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443567 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443406 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443567 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443437 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c42gx\" (UniqueName: \"kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx\") pod \"cab73c91-7f57-4668-adcb-95940723a0b6\" (UID: \"cab73c91-7f57-4668-adcb-95940723a0b6\") " Jul 14 17:43:29.443567 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443540 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:29.443750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443699 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache" (OuterVolumeSpecName: "model-cache") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:29.443750 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443734 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-tmp-dir\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.443826 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.443744 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home" (OuterVolumeSpecName: "home") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:29.445477 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.445443 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:43:29.445653 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.445517 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm" (OuterVolumeSpecName: "dshm") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:29.445653 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.445596 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx" (OuterVolumeSpecName: "kube-api-access-c42gx") pod "cab73c91-7f57-4668-adcb-95940723a0b6" (UID: "cab73c91-7f57-4668-adcb-95940723a0b6"). InnerVolumeSpecName "kube-api-access-c42gx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:43:29.544752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.544715 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cab73c91-7f57-4668-adcb-95940723a0b6-tls-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.544752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.544742 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-home\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.544752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.544751 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-model-cache\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.544752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.544758 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c42gx\" (UniqueName: \"kubernetes.io/projected/cab73c91-7f57-4668-adcb-95940723a0b6-kube-api-access-c42gx\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.545084 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.544768 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cab73c91-7f57-4668-adcb-95940723a0b6-dshm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:29.819178 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.819142 2574 generic.go:356] "Generic (PLEG): container finished" podID="cab73c91-7f57-4668-adcb-95940723a0b6" containerID="e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1" exitCode=0 Jul 14 17:43:29.819353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.819205 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" Jul 14 17:43:29.819353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.819234 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" event={"ID":"cab73c91-7f57-4668-adcb-95940723a0b6","Type":"ContainerDied","Data":"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1"} Jul 14 17:43:29.819353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.819278 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f" event={"ID":"cab73c91-7f57-4668-adcb-95940723a0b6","Type":"ContainerDied","Data":"5ed7f1f0461f6e44d6557451695b9de290fbab2814ae8bdddb10467d68c60358"} Jul 14 17:43:29.819353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.819296 2574 scope.go:117] "RemoveContainer" containerID="e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1" Jul 14 17:43:29.827772 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.827751 2574 scope.go:117] "RemoveContainer" containerID="e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1" Jul 14 17:43:29.827992 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:43:29.827977 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1\": container with ID starting with e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1 not found: ID does not exist" containerID="e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1" Jul 14 17:43:29.828043 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.828000 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1"} err="failed to get container status \"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1\": rpc error: code = NotFound desc = could not find container \"e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1\": container with ID starting with e9364a30259a11a05097849ba6438451eda070b321c35f372571e3b015798bf1 not found: ID does not exist" Jul 14 17:43:29.838052 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.838021 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:43:29.842408 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:29.842386 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1b06a1/test-model-1b06a1-kserve-8d9d99fc5-z857f"] Jul 14 17:43:31.692500 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:31.692463 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cab73c91-7f57-4668-adcb-95940723a0b6" path="/var/lib/kubelet/pods/cab73c91-7f57-4668-adcb-95940723a0b6/volumes" Jul 14 17:43:36.289075 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:36.289018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:36.362969 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:36.362935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:40.029873 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:40.029839 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:43:40.030360 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:40.030067 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" containerID="cri-o://e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866" gracePeriod=30 Jul 14 17:43:40.074721 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:40.074688 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:40.297695 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:40.297597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:41.106927 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:41.106884 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:41.203456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:41.203410 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:42.147203 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:42.147140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:42.226268 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:42.226233 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:43.003719 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:43.003664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:43.112516 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:43.112481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:45.284232 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.284206 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:43:45.389388 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389348 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389401 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqrsm\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389429 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389465 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389494 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389515 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389564 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389541 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389568 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389631 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo\") pod \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\" (UID: \"9bdb40c9-7d56-4c8e-8076-ab27769f50f7\") " Jul 14 17:43:45.389946 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389717 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:45.390107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389951 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.390107 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.389978 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:45.390216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.390112 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:45.390270 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.390219 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:43:45.390858 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.390824 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:45.391910 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.391885 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:43:45.391995 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.391906 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm" (OuterVolumeSpecName: "kube-api-access-lqrsm") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "kube-api-access-lqrsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:43:45.392279 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.392257 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:43:45.392347 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.392281 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bdb40c9-7d56-4c8e-8076-ab27769f50f7" (UID: "9bdb40c9-7d56-4c8e-8076-ab27769f50f7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:43:45.490538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490493 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqrsm\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-kube-api-access-lqrsm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490531 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490540 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490548 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490555 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490563 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490570 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.490839 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.490614 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdb40c9-7d56-4c8e-8076-ab27769f50f7-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:43:45.908832 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.908792 2574 generic.go:356] "Generic (PLEG): container finished" podID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerID="e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866" exitCode=0 Jul 14 17:43:45.908832 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.908840 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" Jul 14 17:43:45.909122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.908881 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerDied","Data":"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866"} Jul 14 17:43:45.909122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.908943 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw" event={"ID":"9bdb40c9-7d56-4c8e-8076-ab27769f50f7","Type":"ContainerDied","Data":"486cd09ac8d1c547d00dbd96f62bf451c00302a60efb79f4fc71ecef1964ef56"} Jul 14 17:43:45.909122 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.908972 2574 scope.go:117] "RemoveContainer" containerID="e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866" Jul 14 17:43:45.918223 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.918202 2574 scope.go:117] "RemoveContainer" containerID="78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730" Jul 14 17:43:45.927029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.927007 2574 scope.go:117] "RemoveContainer" containerID="e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866" Jul 14 17:43:45.927331 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:43:45.927308 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866\": container with ID starting with e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866 not found: ID does not exist" containerID="e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866" Jul 14 17:43:45.927405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.927341 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866"} err="failed to get container status \"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866\": rpc error: code = NotFound desc = could not find container \"e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866\": container with ID starting with e614e0331131fd2cb0714da041d78d8b0f7086c22aca6a270221ffce17af4866 not found: ID does not exist" Jul 14 17:43:45.927405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.927362 2574 scope.go:117] "RemoveContainer" containerID="78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730" Jul 14 17:43:45.927658 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:43:45.927640 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730\": container with ID starting with 78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730 not found: ID does not exist" containerID="78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730" Jul 14 17:43:45.927710 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.927663 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730"} err="failed to get container status \"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730\": rpc error: code = NotFound desc = could not find container \"78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730\": container with ID starting with 78116e60aef25771f075cb44ab8009236ed9f85be4dbec2203e5184f4f37d730 not found: ID does not exist" Jul 14 17:43:45.934006 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.933976 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:43:45.944138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:45.944106 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d44b45-openshift-default-67cbb94c6c-b5cmw"] Jul 14 17:43:47.697229 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:47.697195 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" path="/var/lib/kubelet/pods/9bdb40c9-7d56-4c8e-8076-ab27769f50f7/volumes" Jul 14 17:43:51.144620 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:51.144571 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:43:55.582204 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:43:55.582169 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:03.065802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:03.065717 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:03.209741 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:03.209704 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:06.325530 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:06.325498 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:06.615046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:06.615004 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:06.948556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:06.948472 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:07.256605 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256507 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:44:07.256843 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256831 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:44:07.256895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256844 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:44:07.256895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256856 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:44:07.256895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256862 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:44:07.256895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256872 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cab73c91-7f57-4668-adcb-95940723a0b6" containerName="main" Jul 14 17:44:07.256895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256878 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cab73c91-7f57-4668-adcb-95940723a0b6" containerName="main" Jul 14 17:44:07.257071 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256923 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:44:07.257071 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.256933 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cab73c91-7f57-4668-adcb-95940723a0b6" containerName="main" Jul 14 17:44:07.261232 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.261209 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.263809 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.263788 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-083b58-openshift-default-dockercfg-bpswl\"" Jul 14 17:44:07.277720 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.277686 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:44:07.278002 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.277978 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278090 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278090 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278209 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278098 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278209 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278209 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278191 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278257 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278306 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.278365 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.278352 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfbvr\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.301794 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.301761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:07.378987 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.378949 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.378987 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.378989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379034 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379086 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379145 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379952 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.379952 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.379939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.380155 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.380130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.380222 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.380186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.380286 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.380219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.380286 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.380223 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfbvr\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.382594 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.380872 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.386638 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.382872 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.388151 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.388122 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.392700 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.392680 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.392876 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.392856 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfbvr\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr\") pod \"e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.574760 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.574664 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-083b58-openshift-default-dockercfg-bpswl\"" Jul 14 17:44:07.582174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.582147 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:07.604483 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.604448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:07.725822 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.725796 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:44:07.728444 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:44:07.728415 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode563887b_d7a7_4661_8a3e_117682e504fe.slice/crio-1bfafd0571936e7b76d8abc746126003893e96cb6ac19b389a4e7bbe68d7e853 WatchSource:0}: Error finding container 1bfafd0571936e7b76d8abc746126003893e96cb6ac19b389a4e7bbe68d7e853: Status 404 returned error can't find the container with id 1bfafd0571936e7b76d8abc746126003893e96cb6ac19b389a4e7bbe68d7e853 Jul 14 17:44:07.730490 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.730457 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:07.730558 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.730523 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:07.730615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:07.730552 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:08.015613 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:08.015565 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerStarted","Data":"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701"} Jul 14 17:44:08.015879 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:08.015630 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerStarted","Data":"1bfafd0571936e7b76d8abc746126003893e96cb6ac19b389a4e7bbe68d7e853"} Jul 14 17:44:08.044588 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:08.044519 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podStartSLOduration=1.044503583 podStartE2EDuration="1.044503583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:44:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:44:08.042038462 +0000 UTC m=+2341.564979239" watchObservedRunningTime="2026-07-14 17:44:08.044503583 +0000 UTC m=+2341.567444355" Jul 14 17:44:08.582428 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:08.582383 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:08.587113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:08.587086 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:09.020837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:09.020748 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:09.021689 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:09.021669 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:10.231743 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:10.231701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:10.373569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:10.373531 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:10.674101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:10.674067 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:11.125824 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:11.125792 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:11.232396 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:11.232364 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:14.693338 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:14.693300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:15.033106 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:15.033015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:15.343098 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:15.343063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:16.005868 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:16.005831 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:18.163076 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:18.163040 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:18.300863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:18.300826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:18.507281 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:18.507183 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:18.612291 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:18.612256 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:19.282503 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:19.282456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:19.415089 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:19.415056 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:22.708184 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:22.708106 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:23.093055 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.093018 2574 generic.go:356] "Generic (PLEG): container finished" podID="e563887b-d7a7-4661-8a3e-117682e504fe" containerID="09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701" exitCode=0 Jul 14 17:44:23.093231 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.093091 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerDied","Data":"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701"} Jul 14 17:44:23.093486 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.093470 2574 scope.go:117] "RemoveContainer" containerID="09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701" Jul 14 17:44:23.094512 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.094485 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:23.094601 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.094568 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:23.094644 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:23.094634 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:44:24.020721 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:24.020676 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:24.103491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:24.103446 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerStarted","Data":"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599"} Jul 14 17:44:24.582588 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:24.582536 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:24.584167 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:24.584131 2574 patch_prober.go:28] interesting pod/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 17:44:24.584414 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:24.584382 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 17:44:25.583024 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:25.582980 2574 patch_prober.go:28] interesting pod/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 17:44:25.583599 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:25.583048 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 17:44:26.583194 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:26.583152 2574 patch_prober.go:28] interesting pod/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 17:44:26.583826 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:26.583790 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 17:44:27.582725 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:27.582683 2574 patch_prober.go:28] interesting pod/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 14 17:44:27.583050 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:27.583009 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 14 17:44:28.586679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:28.586646 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:28.587049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:28.586922 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:28.587537 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:28.587523 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:44:31.443770 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:31.443733 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:31.540904 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:31.540874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:42.364723 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:42.364668 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:42.457789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:42.457758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:42.763970 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:42.763885 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:45.851555 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:45.851514 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:52.397258 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:52.397213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:52.497275 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:52.497237 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:56.517353 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:56.517321 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:44:56.517843 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:56.517539 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" containerID="cri-o://cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599" gracePeriod=30 Jul 14 17:44:56.538557 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:56.538521 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:56.809215 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:56.809119 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:44:58.589199 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:58.589163 2574 patch_prober.go:28] interesting pod/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 14 17:44:58.589608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:44:58.589229 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:45:01.765208 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.765178 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:45:01.907105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907068 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907105 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907108 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907135 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907164 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907182 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907250 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907282 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907350 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907327 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfbvr\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907354 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data\") pod \"e563887b-d7a7-4661-8a3e-117682e504fe\" (UID: \"e563887b-d7a7-4661-8a3e-117682e504fe\") " Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907450 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907554 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907599 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907690 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907710 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907723 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:01.907976 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.907924 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:45:01.909112 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.909062 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:45:01.909639 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.909612 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:45:01.909764 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.909738 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:45:01.909829 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.909809 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:45:01.909982 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:01.909959 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr" (OuterVolumeSpecName: "kube-api-access-dfbvr") pod "e563887b-d7a7-4661-8a3e-117682e504fe" (UID: "e563887b-d7a7-4661-8a3e-117682e504fe"). InnerVolumeSpecName "kube-api-access-dfbvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:45:02.008329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008269 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.008329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008320 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.008329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008330 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e563887b-d7a7-4661-8a3e-117682e504fe-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.008329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008338 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e563887b-d7a7-4661-8a3e-117682e504fe-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.008329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008347 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfbvr\" (UniqueName: \"kubernetes.io/projected/e563887b-d7a7-4661-8a3e-117682e504fe-kube-api-access-dfbvr\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.008680 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.008355 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e563887b-d7a7-4661-8a3e-117682e504fe-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:45:02.290553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.290452 2574 generic.go:356] "Generic (PLEG): container finished" podID="e563887b-d7a7-4661-8a3e-117682e504fe" containerID="cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599" exitCode=0 Jul 14 17:45:02.290553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.290503 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" Jul 14 17:45:02.290553 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.290538 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerDied","Data":"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599"} Jul 14 17:45:02.290830 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.290609 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v" event={"ID":"e563887b-d7a7-4661-8a3e-117682e504fe","Type":"ContainerDied","Data":"1bfafd0571936e7b76d8abc746126003893e96cb6ac19b389a4e7bbe68d7e853"} Jul 14 17:45:02.290830 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.290626 2574 scope.go:117] "RemoveContainer" containerID="cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599" Jul 14 17:45:02.302282 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.302260 2574 scope.go:117] "RemoveContainer" containerID="09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701" Jul 14 17:45:02.310984 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.310961 2574 scope.go:117] "RemoveContainer" containerID="cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599" Jul 14 17:45:02.311253 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:45:02.311234 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599\": container with ID starting with cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599 not found: ID does not exist" containerID="cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599" Jul 14 17:45:02.311320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.311260 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599"} err="failed to get container status \"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599\": rpc error: code = NotFound desc = could not find container \"cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599\": container with ID starting with cedc1be2ebe2e03374c95e730e079e60b59fa31bf6f6c3095456670725b12599 not found: ID does not exist" Jul 14 17:45:02.311320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.311277 2574 scope.go:117] "RemoveContainer" containerID="09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701" Jul 14 17:45:02.311510 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:45:02.311492 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701\": container with ID starting with 09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701 not found: ID does not exist" containerID="09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701" Jul 14 17:45:02.311551 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.311516 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701"} err="failed to get container status \"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701\": rpc error: code = NotFound desc = could not find container \"09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701\": container with ID starting with 09898874859b39570b932945e9abe0d627695c8cb469add2fe406a21c65ca701 not found: ID does not exist" Jul 14 17:45:02.317302 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.317274 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:45:02.322380 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:02.322352 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-083b58-openshift-default-6bbd967df5-xw29v"] Jul 14 17:45:03.692428 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:03.692386 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" path="/var/lib/kubelet/pods/e563887b-d7a7-4661-8a3e-117682e504fe/volumes" Jul 14 17:45:07.482357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:07.482309 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:07.777603 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:07.777498 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:45:07.780834 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:07.780811 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:45:12.015340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:12.015304 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:18.530748 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:18.530711 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:19.235406 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:19.235373 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:19.326113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:19.326080 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:22.704479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:22.704439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:22.966993 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:22.966899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:23.305615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.305500 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:23.592921 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.592866 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:45:23.593297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593263 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:45:23.593297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593290 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:45:23.593494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593320 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:45:23.593494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593329 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:45:23.593494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593416 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:45:23.593494 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.593433 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bdb40c9-7d56-4c8e-8076-ab27769f50f7" containerName="istio-proxy" Jul 14 17:45:23.596605 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.596586 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.599897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.599871 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-246d80-openshift-default-dockercfg-crf46\"" Jul 14 17:45:23.613666 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.613630 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:45:23.627566 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.627537 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:23.703036 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703042 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703092 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703147 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703186 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703239 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703503 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703272 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmrsp\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.703503 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.703310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.804549 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804559 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmrsp\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804624 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804678 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804725 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804747 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804827 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.805007 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805378 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.804859 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805378 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.805088 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805378 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.805066 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805378 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.805332 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.805526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.805412 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.807018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.806994 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.807254 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.807235 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.814348 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.814322 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmrsp\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.814464 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.814370 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token\") pod \"e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:23.836047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.836014 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:23.907233 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:23.907139 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:24.050317 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.050285 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:45:24.052036 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:45:24.052012 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90badcf8_5bf3_4fa8_bc4c_b11b2494b887.slice/crio-85f836c933ee62a695de30b9695b7fd64f9e0c639500fbf0dd55f08d704cea0b WatchSource:0}: Error finding container 85f836c933ee62a695de30b9695b7fd64f9e0c639500fbf0dd55f08d704cea0b: Status 404 returned error can't find the container with id 85f836c933ee62a695de30b9695b7fd64f9e0c639500fbf0dd55f08d704cea0b Jul 14 17:45:24.054235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.054200 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:24.054335 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.054284 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:24.054335 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.054325 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:24.398812 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.398775 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerStarted","Data":"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e"} Jul 14 17:45:24.399038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.398836 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerStarted","Data":"85f836c933ee62a695de30b9695b7fd64f9e0c639500fbf0dd55f08d704cea0b"} Jul 14 17:45:24.443032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.442976 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podStartSLOduration=1.442960715 podStartE2EDuration="1.442960715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:45:24.441000471 +0000 UTC m=+2417.963941246" watchObservedRunningTime="2026-07-14 17:45:24.442960715 +0000 UTC m=+2417.965901488" Jul 14 17:45:24.907859 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:24.907820 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:25.912335 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:25.912295 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:26.376534 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.376483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:26.410319 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.410284 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:26.411162 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.411144 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:26.477388 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.477352 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:26.586245 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.586211 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:26.761260 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:26.761165 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:27.174894 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:27.174857 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:27.292473 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:27.292438 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:30.903641 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:30.903534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:31.262321 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:31.262232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:31.571427 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:31.571330 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:31.667408 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:31.667370 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:34.492869 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:34.492836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:34.593747 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:34.593713 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:34.700863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:34.700822 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:35.034456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:35.034424 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:35.246056 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:35.246020 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:35.344569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:35.344534 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:38.950877 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:38.950837 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:40.480064 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.480029 2574 generic.go:356] "Generic (PLEG): container finished" podID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerID="6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e" exitCode=0 Jul 14 17:45:40.480510 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.480102 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerDied","Data":"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e"} Jul 14 17:45:40.480556 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.480518 2574 scope.go:117] "RemoveContainer" containerID="6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e" Jul 14 17:45:40.481509 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.481482 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:40.481635 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.481549 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:40.481635 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:40.481606 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 17:45:41.411031 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:41.410993 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:41.503714 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:41.503677 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerStarted","Data":"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b"} Jul 14 17:45:41.907679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:41.907637 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:41.909000 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:41.908971 2574 patch_prober.go:28] interesting pod/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 17:45:41.909138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:41.909022 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 17:45:42.907554 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:42.907513 2574 patch_prober.go:28] interesting pod/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 17:45:42.907983 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:42.907602 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 17:45:43.908451 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:43.908409 2574 patch_prober.go:28] interesting pod/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 17:45:43.908893 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:43.908472 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 17:45:44.908234 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:44.908196 2574 patch_prober.go:28] interesting pod/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 14 17:45:44.908440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:44.908257 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 14 17:45:45.912319 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:45.912281 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:45.912800 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:45.912565 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:45.913297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:45.913277 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:45:48.161046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:48.161006 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:48.272016 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:48.271983 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:52.003145 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:52.003108 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:57.563223 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:57.563187 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:45:57.661569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:45:57.661535 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:46:19.150591 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.150530 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:46:19.152034 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.152007 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e563887b-d7a7-4661-8a3e-117682e504fe" containerName="istio-proxy" Jul 14 17:46:19.156256 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.156234 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.163115 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.163089 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"kube-root-ca.crt\"" Jul 14 17:46:19.163457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.163441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"auth-test-model-df3dc5-kserve-self-signed-certs\"" Jul 14 17:46:19.163555 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.163495 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"default-dockercfg-vhsbb\"" Jul 14 17:46:19.163655 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.163564 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-df3dc5\"/\"openshift-service-ca.crt\"" Jul 14 17:46:19.171501 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.171479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:46:19.202329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.202329 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.202533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202377 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8dc7\" (UniqueName: \"kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.202533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.202533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202432 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.202533 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.202452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303327 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303287 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8dc7\" (UniqueName: \"kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303403 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303428 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303511 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303455 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303838 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303767 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303899 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303852 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.303899 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.303877 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.305657 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.305639 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.305840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.305823 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.314924 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.314902 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8dc7\" (UniqueName: \"kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7\") pod \"auth-test-model-df3dc5-kserve-77954878f9-8nm46\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.466338 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.466247 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:19.606402 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.606370 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:46:19.607993 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:46:19.607966 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59778b03_facc_498c_84ba_224ee53421dc.slice/crio-d657ad165fb0614ca60a1cf42068b062488d54c2a0324b985d01d9d04620b9f0 WatchSource:0}: Error finding container d657ad165fb0614ca60a1cf42068b062488d54c2a0324b985d01d9d04620b9f0: Status 404 returned error can't find the container with id d657ad165fb0614ca60a1cf42068b062488d54c2a0324b985d01d9d04620b9f0 Jul 14 17:46:19.692881 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.692838 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" event={"ID":"59778b03-facc-498c-84ba-224ee53421dc","Type":"ContainerStarted","Data":"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611"} Jul 14 17:46:19.693046 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.692902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" event={"ID":"59778b03-facc-498c-84ba-224ee53421dc","Type":"ContainerStarted","Data":"d657ad165fb0614ca60a1cf42068b062488d54c2a0324b985d01d9d04620b9f0"} Jul 14 17:46:19.712762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:19.712714 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" podStartSLOduration=0.712696975 podStartE2EDuration="712.696975ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:46:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:46:19.712190277 +0000 UTC m=+2473.235131050" watchObservedRunningTime="2026-07-14 17:46:19.712696975 +0000 UTC m=+2473.235637747" Jul 14 17:46:20.839009 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:20.838977 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:46:24.548196 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:24.548162 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:46:29.466431 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:29.466388 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:29.466881 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:29.466445 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:29.480526 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:29.480489 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:29.752180 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:29.752094 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:46:35.715449 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:46:35.715414 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:10.178840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:10.178808 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:26.259390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:26.259351 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:27.128538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:27.128500 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:48:27.146789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:27.146746 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:42.148721 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.148676 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" podUID="59778b03-facc-498c-84ba-224ee53421dc" containerName="main" containerID="cri-o://aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611" gracePeriod=45 Jul 14 17:48:42.403770 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.403687 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:48:42.404206 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.404182 2574 generic.go:356] "Generic (PLEG): container finished" podID="59778b03-facc-498c-84ba-224ee53421dc" containerID="aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611" exitCode=0 Jul 14 17:48:42.404274 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.404237 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" event={"ID":"59778b03-facc-498c-84ba-224ee53421dc","Type":"ContainerDied","Data":"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611"} Jul 14 17:48:42.404326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.404269 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" event={"ID":"59778b03-facc-498c-84ba-224ee53421dc","Type":"ContainerDied","Data":"d657ad165fb0614ca60a1cf42068b062488d54c2a0324b985d01d9d04620b9f0"} Jul 14 17:48:42.404326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.404296 2574 scope.go:117] "RemoveContainer" containerID="aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611" Jul 14 17:48:42.412980 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.412958 2574 scope.go:117] "RemoveContainer" containerID="aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611" Jul 14 17:48:42.413322 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:48:42.413302 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611\": container with ID starting with aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611 not found: ID does not exist" containerID="aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611" Jul 14 17:48:42.413390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.413330 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611"} err="failed to get container status \"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611\": rpc error: code = NotFound desc = could not find container \"aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611\": container with ID starting with aec663aa2060ae993bc1bbdc1ee5df6e10621e25d6bdf1f71a89988ef4157611 not found: ID does not exist" Jul 14 17:48:42.570806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570772 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570822 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570851 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8dc7\" (UniqueName: \"kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570881 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570914 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.570935 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home\") pod \"59778b03-facc-498c-84ba-224ee53421dc\" (UID: \"59778b03-facc-498c-84ba-224ee53421dc\") " Jul 14 17:48:42.571288 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.571263 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:48:42.571342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.571279 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home" (OuterVolumeSpecName: "home") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:48:42.571342 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.571291 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:48:42.573246 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.573214 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:48:42.573246 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.573222 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7" (OuterVolumeSpecName: "kube-api-access-b8dc7") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "kube-api-access-b8dc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:48:42.573390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.573275 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm" (OuterVolumeSpecName: "dshm") pod "59778b03-facc-498c-84ba-224ee53421dc" (UID: "59778b03-facc-498c-84ba-224ee53421dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.671925 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-home\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.671956 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-dshm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.671965 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59778b03-facc-498c-84ba-224ee53421dc-tls-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.671975 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8dc7\" (UniqueName: \"kubernetes.io/projected/59778b03-facc-498c-84ba-224ee53421dc-kube-api-access-b8dc7\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.671989 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-model-cache\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:42.672018 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:42.672001 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59778b03-facc-498c-84ba-224ee53421dc-tmp-dir\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:48:43.113440 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.113406 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:43.212615 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.212555 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:43.245192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.245158 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:48:43.245483 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.245471 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59778b03-facc-498c-84ba-224ee53421dc" containerName="main" Jul 14 17:48:43.245531 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.245484 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="59778b03-facc-498c-84ba-224ee53421dc" containerName="main" Jul 14 17:48:43.245567 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.245544 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="59778b03-facc-498c-84ba-224ee53421dc" containerName="main" Jul 14 17:48:43.249833 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.249810 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.252841 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.252823 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"kube-root-ca.crt\"" Jul 14 17:48:43.254289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.254262 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"openshift-service-ca.crt\"" Jul 14 17:48:43.254403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.254295 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"sub-test-model-246d80-kserve-self-signed-certs\"" Jul 14 17:48:43.254403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.254354 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"default-dockercfg-45tmv\"" Jul 14 17:48:43.258798 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.258774 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:48:43.312732 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.312697 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:48:43.379160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379070 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.379160 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.379357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.379357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.379357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379206 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.379357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.379246 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9ppr\" (UniqueName: \"kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.411064 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.411022 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46" Jul 14 17:48:43.438114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.438054 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:48:43.440630 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.440603 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-df3dc5/auth-test-model-df3dc5-kserve-77954878f9-8nm46"] Jul 14 17:48:43.479918 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.479880 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.479918 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.479918 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480161 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.479947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480161 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480092 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9ppr\" (UniqueName: \"kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480269 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480207 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480269 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480391 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480379 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480551 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480530 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.480661 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.480641 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.482276 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.482252 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.482477 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.482459 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.492512 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.492483 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9ppr\" (UniqueName: \"kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr\") pod \"sub-test-model-246d80-kserve-7b555646c9-btpzp\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.560546 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.560503 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:43.694140 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.694104 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59778b03-facc-498c-84ba-224ee53421dc" path="/var/lib/kubelet/pods/59778b03-facc-498c-84ba-224ee53421dc/volumes" Jul 14 17:48:43.701140 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:43.701112 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:48:43.703284 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:48:43.703259 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b0845b3_dbbd_4a0d_b916_ec854fc810e0.slice/crio-fcc4f24d71275afd952a7caff8b475196c978f4cd81862b967cfff4b63014440 WatchSource:0}: Error finding container fcc4f24d71275afd952a7caff8b475196c978f4cd81862b967cfff4b63014440: Status 404 returned error can't find the container with id fcc4f24d71275afd952a7caff8b475196c978f4cd81862b967cfff4b63014440 Jul 14 17:48:44.424844 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:44.424806 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" event={"ID":"1b0845b3-dbbd-4a0d-b916-ec854fc810e0","Type":"ContainerStarted","Data":"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242"} Jul 14 17:48:44.425249 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:44.424855 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" event={"ID":"1b0845b3-dbbd-4a0d-b916-ec854fc810e0","Type":"ContainerStarted","Data":"fcc4f24d71275afd952a7caff8b475196c978f4cd81862b967cfff4b63014440"} Jul 14 17:48:44.446299 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:44.446241 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" podStartSLOduration=1.4462248469999999 podStartE2EDuration="1.446224847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:48:44.444857891 +0000 UTC m=+2617.967798665" watchObservedRunningTime="2026-07-14 17:48:44.446224847 +0000 UTC m=+2617.969165620" Jul 14 17:48:53.561297 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:53.561244 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:53.561741 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:53.561318 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:53.574024 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:53.573996 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:48:54.488853 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:48:54.488817 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:49:04.829951 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:04.829911 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:49:06.363779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:06.363743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:49:22.034711 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:22.034673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:49:51.122608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:51.122552 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:49:51.144936 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:51.144905 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:49:51.980684 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:51.980646 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:49:59.131863 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:49:59.131780 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:06.142456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.142389 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" podUID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" containerName="main" containerID="cri-o://0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242" gracePeriod=45 Jul 14 17:50:06.392923 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.392862 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:50:06.516538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516485 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.516538 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516537 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.516806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516714 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.516806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516739 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9ppr\" (UniqueName: \"kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.516806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516782 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.516975 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.516818 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir\") pod \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\" (UID: \"1b0845b3-dbbd-4a0d-b916-ec854fc810e0\") " Jul 14 17:50:06.517085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517045 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:50:06.517186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517047 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home" (OuterVolumeSpecName: "home") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:50:06.517186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517082 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:50:06.517442 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517421 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-model-cache\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.517480 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517452 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tmp-dir\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.517480 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.517467 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-home\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.518779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.518754 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm" (OuterVolumeSpecName: "dshm") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:50:06.518885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.518802 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:50:06.518885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.518806 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr" (OuterVolumeSpecName: "kube-api-access-q9ppr") pod "1b0845b3-dbbd-4a0d-b916-ec854fc810e0" (UID: "1b0845b3-dbbd-4a0d-b916-ec854fc810e0"). InnerVolumeSpecName "kube-api-access-q9ppr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:50:06.618231 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.618190 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-tls-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.618231 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.618222 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-dshm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.618231 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.618231 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9ppr\" (UniqueName: \"kubernetes.io/projected/1b0845b3-dbbd-4a0d-b916-ec854fc810e0-kube-api-access-q9ppr\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:50:06.646466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.646385 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:50:06.646746 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.646733 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" containerName="main" Jul 14 17:50:06.646797 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.646748 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" containerName="main" Jul 14 17:50:06.646833 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.646818 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" containerName="main" Jul 14 17:50:06.650054 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.650032 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.652870 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.652850 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-246d80\"/\"rate-test-model-246d80-kserve-self-signed-certs\"" Jul 14 17:50:06.662638 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.662610 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:50:06.702635 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.702601 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:06.718851 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.718813 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.719005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.718863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.719005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.718911 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.719005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.718993 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lmrz\" (UniqueName: \"kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.719114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.719055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.719114 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.719090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820111 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820072 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lmrz\" (UniqueName: \"kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820111 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820121 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820154 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820397 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820234 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820649 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820619 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820716 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820693 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.820997 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.820974 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.823340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.823315 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.823458 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.823405 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.835540 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.835508 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lmrz\" (UniqueName: \"kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz\") pod \"rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:06.836495 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.836460 2574 generic.go:356] "Generic (PLEG): container finished" podID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" containerID="0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242" exitCode=0 Jul 14 17:50:06.836656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.836501 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" Jul 14 17:50:06.836656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.836508 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" event={"ID":"1b0845b3-dbbd-4a0d-b916-ec854fc810e0","Type":"ContainerDied","Data":"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242"} Jul 14 17:50:06.836656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.836561 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp" event={"ID":"1b0845b3-dbbd-4a0d-b916-ec854fc810e0","Type":"ContainerDied","Data":"fcc4f24d71275afd952a7caff8b475196c978f4cd81862b967cfff4b63014440"} Jul 14 17:50:06.836656 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.836620 2574 scope.go:117] "RemoveContainer" containerID="0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242" Jul 14 17:50:06.845663 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.845646 2574 scope.go:117] "RemoveContainer" containerID="0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242" Jul 14 17:50:06.845949 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:50:06.845930 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242\": container with ID starting with 0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242 not found: ID does not exist" containerID="0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242" Jul 14 17:50:06.846001 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.845958 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242"} err="failed to get container status \"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242\": rpc error: code = NotFound desc = could not find container \"0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242\": container with ID starting with 0945a8c33348f3165c9f0675eedaf19f3a453986f9c3c0761c00698837acf242 not found: ID does not exist" Jul 14 17:50:06.865429 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.865393 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:50:06.871024 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.870994 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/sub-test-model-246d80-kserve-7b555646c9-btpzp"] Jul 14 17:50:06.959068 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:06.958965 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:07.130264 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.130235 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:50:07.132055 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:50:07.132027 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5a0a060_e6a1_4e8c_9148_5c69d077d01e.slice/crio-4bda76bb83159ca3562b0efec7af532e321a835fafbc5e150cb8c411ce3f9fb1 WatchSource:0}: Error finding container 4bda76bb83159ca3562b0efec7af532e321a835fafbc5e150cb8c411ce3f9fb1: Status 404 returned error can't find the container with id 4bda76bb83159ca3562b0efec7af532e321a835fafbc5e150cb8c411ce3f9fb1 Jul 14 17:50:07.692383 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.692352 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b0845b3-dbbd-4a0d-b916-ec854fc810e0" path="/var/lib/kubelet/pods/1b0845b3-dbbd-4a0d-b916-ec854fc810e0/volumes" Jul 14 17:50:07.809621 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.809593 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:50:07.812842 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.812818 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tj95b_e1090c57-e98d-4da2-bcc1-23cae48d366e/ovn-acl-logging/0.log" Jul 14 17:50:07.843381 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.843347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" event={"ID":"a5a0a060-e6a1-4e8c-9148-5c69d077d01e","Type":"ContainerStarted","Data":"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46"} Jul 14 17:50:07.843569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.843389 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" event={"ID":"a5a0a060-e6a1-4e8c-9148-5c69d077d01e","Type":"ContainerStarted","Data":"4bda76bb83159ca3562b0efec7af532e321a835fafbc5e150cb8c411ce3f9fb1"} Jul 14 17:50:07.865589 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:07.865535 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" podStartSLOduration=1.865518723 podStartE2EDuration="1.865518723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:50:07.86520694 +0000 UTC m=+2701.388147710" watchObservedRunningTime="2026-07-14 17:50:07.865518723 +0000 UTC m=+2701.388459496" Jul 14 17:50:16.959491 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:16.959440 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:16.959933 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:16.959511 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:16.972247 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:16.972216 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:17.905978 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:17.905942 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:50:40.747670 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:40.747631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:49.262218 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:49.262178 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:49.375374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:49.375340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:59.002882 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:59.002832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:50:59.157844 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:50:59.157796 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:01.273746 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:01.273673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:23.586534 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:23.586496 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:51:23.633987 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:23.633955 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:24.354557 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:24.354519 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:28.603118 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.603036 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961553968"] Jul 14 17:51:28.606313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.606294 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:28.609126 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.609107 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-8744v\"" Jul 14 17:51:28.614495 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.614471 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961553968"] Jul 14 17:51:28.745851 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.745815 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfkzq\" (UniqueName: \"kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq\") pod \"test-curl-7335-140017961553968\" (UID: \"eee25dc3-6f62-4fe7-b207-579307afff8b\") " pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:28.847275 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.847235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfkzq\" (UniqueName: \"kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq\") pod \"test-curl-7335-140017961553968\" (UID: \"eee25dc3-6f62-4fe7-b207-579307afff8b\") " pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:28.857320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.857255 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfkzq\" (UniqueName: \"kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq\") pod \"test-curl-7335-140017961553968\" (UID: \"eee25dc3-6f62-4fe7-b207-579307afff8b\") " pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:28.915457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:28.915428 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:29.051189 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:29.051159 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961553968"] Jul 14 17:51:29.052784 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:51:29.052759 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeee25dc3_6f62_4fe7_b207_579307afff8b.slice/crio-70674ed2fbf0f2a2b5cf33d04d5f7ff71105824361fa58dd75ed7c7cde1c6c77 WatchSource:0}: Error finding container 70674ed2fbf0f2a2b5cf33d04d5f7ff71105824361fa58dd75ed7c7cde1c6c77: Status 404 returned error can't find the container with id 70674ed2fbf0f2a2b5cf33d04d5f7ff71105824361fa58dd75ed7c7cde1c6c77 Jul 14 17:51:29.054562 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:29.054540 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:51:29.265773 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:29.265674 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961553968" event={"ID":"eee25dc3-6f62-4fe7-b207-579307afff8b","Type":"ContainerStarted","Data":"70674ed2fbf0f2a2b5cf33d04d5f7ff71105824361fa58dd75ed7c7cde1c6c77"} Jul 14 17:51:30.272779 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:30.272740 2574 generic.go:356] "Generic (PLEG): container finished" podID="eee25dc3-6f62-4fe7-b207-579307afff8b" containerID="9f1a1b65b70845197539510c7c5ff29b2e7bcadb1484365eaf8f253dea7d6e62" exitCode=0 Jul 14 17:51:30.273185 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:30.272826 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961553968" event={"ID":"eee25dc3-6f62-4fe7-b207-579307afff8b","Type":"ContainerDied","Data":"9f1a1b65b70845197539510c7c5ff29b2e7bcadb1484365eaf8f253dea7d6e62"} Jul 14 17:51:31.416798 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.416769 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:31.439237 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.439213 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961553968_eee25dc3-6f62-4fe7-b207-579307afff8b/test-curl-7335-140017961553968/0.log" Jul 14 17:51:31.464847 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.464814 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961553968"] Jul 14 17:51:31.469045 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.469020 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961553968"] Jul 14 17:51:31.469171 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.469088 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfkzq\" (UniqueName: \"kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq\") pod \"eee25dc3-6f62-4fe7-b207-579307afff8b\" (UID: \"eee25dc3-6f62-4fe7-b207-579307afff8b\") " Jul 14 17:51:31.471214 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.471190 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq" (OuterVolumeSpecName: "kube-api-access-cfkzq") pod "eee25dc3-6f62-4fe7-b207-579307afff8b" (UID: "eee25dc3-6f62-4fe7-b207-579307afff8b"). InnerVolumeSpecName "kube-api-access-cfkzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:31.570816 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.570729 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfkzq\" (UniqueName: \"kubernetes.io/projected/eee25dc3-6f62-4fe7-b207-579307afff8b-kube-api-access-cfkzq\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:31.634713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.634673 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:31.635079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.635061 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee25dc3-6f62-4fe7-b207-579307afff8b" containerName="test-curl-7335-140017961553968" Jul 14 17:51:31.635165 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.635081 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee25dc3-6f62-4fe7-b207-579307afff8b" containerName="test-curl-7335-140017961553968" Jul 14 17:51:31.635219 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.635173 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee25dc3-6f62-4fe7-b207-579307afff8b" containerName="test-curl-7335-140017961553968" Jul 14 17:51:31.638005 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.637985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:31.645746 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.645723 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:31.692416 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.692383 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee25dc3-6f62-4fe7-b207-579307afff8b" path="/var/lib/kubelet/pods/eee25dc3-6f62-4fe7-b207-579307afff8b/volumes" Jul 14 17:51:31.772390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.772350 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kfqx\" (UniqueName: \"kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx\") pod \"test-curl-7335-140017961499216\" (UID: \"e1593f09-33e4-4a83-822d-a15350257f86\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:31.873061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.873023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kfqx\" (UniqueName: \"kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx\") pod \"test-curl-7335-140017961499216\" (UID: \"e1593f09-33e4-4a83-822d-a15350257f86\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:31.884608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.884567 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kfqx\" (UniqueName: \"kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx\") pod \"test-curl-7335-140017961499216\" (UID: \"e1593f09-33e4-4a83-822d-a15350257f86\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:31.947398 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:31.947364 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:32.086319 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:32.086283 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:32.089484 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:51:32.089457 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1593f09_33e4_4a83_822d_a15350257f86.slice/crio-3c0b23a2dc4b16fae692d257365db659fb471cf5386e9d881f17ab01d16ec55e WatchSource:0}: Error finding container 3c0b23a2dc4b16fae692d257365db659fb471cf5386e9d881f17ab01d16ec55e: Status 404 returned error can't find the container with id 3c0b23a2dc4b16fae692d257365db659fb471cf5386e9d881f17ab01d16ec55e Jul 14 17:51:32.287292 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:32.287265 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961553968" Jul 14 17:51:32.287479 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:32.287275 2574 scope.go:117] "RemoveContainer" containerID="9f1a1b65b70845197539510c7c5ff29b2e7bcadb1484365eaf8f253dea7d6e62" Jul 14 17:51:32.288755 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:32.288702 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961499216" event={"ID":"e1593f09-33e4-4a83-822d-a15350257f86","Type":"ContainerStarted","Data":"3c0b23a2dc4b16fae692d257365db659fb471cf5386e9d881f17ab01d16ec55e"} Jul 14 17:51:33.295908 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:33.295870 2574 generic.go:356] "Generic (PLEG): container finished" podID="e1593f09-33e4-4a83-822d-a15350257f86" containerID="7b951960a271f59c3109ca3418bda07da38635af8b31f8eb17f9f7568c658328" exitCode=0 Jul 14 17:51:33.296328 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:33.295949 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961499216" event={"ID":"e1593f09-33e4-4a83-822d-a15350257f86","Type":"ContainerDied","Data":"7b951960a271f59c3109ca3418bda07da38635af8b31f8eb17f9f7568c658328"} Jul 14 17:51:34.441374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.441347 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:34.465943 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.465909 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961499216_e1593f09-33e4-4a83-822d-a15350257f86/test-curl-7335-140017961499216/0.log" Jul 14 17:51:34.492640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.492608 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:34.494989 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.494963 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kfqx\" (UniqueName: \"kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx\") pod \"e1593f09-33e4-4a83-822d-a15350257f86\" (UID: \"e1593f09-33e4-4a83-822d-a15350257f86\") " Jul 14 17:51:34.496192 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.496169 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:34.497235 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.497211 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx" (OuterVolumeSpecName: "kube-api-access-7kfqx") pod "e1593f09-33e4-4a83-822d-a15350257f86" (UID: "e1593f09-33e4-4a83-822d-a15350257f86"). InnerVolumeSpecName "kube-api-access-7kfqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:34.596186 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.596154 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kfqx\" (UniqueName: \"kubernetes.io/projected/e1593f09-33e4-4a83-822d-a15350257f86-kube-api-access-7kfqx\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:34.616496 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.616462 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:34.616837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.616823 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1593f09-33e4-4a83-822d-a15350257f86" containerName="test-curl-7335-140017961499216" Jul 14 17:51:34.616885 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.616838 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1593f09-33e4-4a83-822d-a15350257f86" containerName="test-curl-7335-140017961499216" Jul 14 17:51:34.616921 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.616895 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1593f09-33e4-4a83-822d-a15350257f86" containerName="test-curl-7335-140017961499216" Jul 14 17:51:34.619776 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.619760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:34.628181 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.628155 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:34.697256 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.697225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhg67\" (UniqueName: \"kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67\") pod \"test-curl-7335-140017961499216\" (UID: \"a1888a10-2980-4d74-81bd-f0005ef4f0c0\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:34.797984 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.797943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhg67\" (UniqueName: \"kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67\") pod \"test-curl-7335-140017961499216\" (UID: \"a1888a10-2980-4d74-81bd-f0005ef4f0c0\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:34.807405 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.807375 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhg67\" (UniqueName: \"kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67\") pod \"test-curl-7335-140017961499216\" (UID: \"a1888a10-2980-4d74-81bd-f0005ef4f0c0\") " pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:34.928886 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:34.928793 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:35.066061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.066025 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:35.069076 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:51:35.069047 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1888a10_2980_4d74_81bd_f0005ef4f0c0.slice/crio-de7820bbfef9a1736db3320fa8aaaabcd8a56ecfa15698c841b83e98f8bed945 WatchSource:0}: Error finding container de7820bbfef9a1736db3320fa8aaaabcd8a56ecfa15698c841b83e98f8bed945: Status 404 returned error can't find the container with id de7820bbfef9a1736db3320fa8aaaabcd8a56ecfa15698c841b83e98f8bed945 Jul 14 17:51:35.307319 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.307203 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961499216" event={"ID":"a1888a10-2980-4d74-81bd-f0005ef4f0c0","Type":"ContainerStarted","Data":"de7820bbfef9a1736db3320fa8aaaabcd8a56ecfa15698c841b83e98f8bed945"} Jul 14 17:51:35.308420 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.308394 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:35.308535 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.308400 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c0b23a2dc4b16fae692d257365db659fb471cf5386e9d881f17ab01d16ec55e" Jul 14 17:51:35.320645 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.320607 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7335-140017961499216" oldPodUID="e1593f09-33e4-4a83-822d-a15350257f86" podUID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" Jul 14 17:51:35.692690 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:35.692661 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1593f09-33e4-4a83-822d-a15350257f86" path="/var/lib/kubelet/pods/e1593f09-33e4-4a83-822d-a15350257f86/volumes" Jul 14 17:51:36.315898 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:36.315862 2574 generic.go:356] "Generic (PLEG): container finished" podID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" containerID="bef4dcc8cf3ebae6a969785ecfa8b8b8fb2525855c248ae90c30b552ef322160" exitCode=0 Jul 14 17:51:36.316090 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:36.315950 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961499216" event={"ID":"a1888a10-2980-4d74-81bd-f0005ef4f0c0","Type":"ContainerDied","Data":"bef4dcc8cf3ebae6a969785ecfa8b8b8fb2525855c248ae90c30b552ef322160"} Jul 14 17:51:37.458059 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.458034 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:37.480785 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.480760 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961499216_a1888a10-2980-4d74-81bd-f0005ef4f0c0/test-curl-7335-140017961499216/0.log" Jul 14 17:51:37.505527 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.505491 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:37.510922 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.510893 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961499216"] Jul 14 17:51:37.524147 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.524119 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhg67\" (UniqueName: \"kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67\") pod \"a1888a10-2980-4d74-81bd-f0005ef4f0c0\" (UID: \"a1888a10-2980-4d74-81bd-f0005ef4f0c0\") " Jul 14 17:51:37.526179 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.526150 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67" (OuterVolumeSpecName: "kube-api-access-bhg67") pod "a1888a10-2980-4d74-81bd-f0005ef4f0c0" (UID: "a1888a10-2980-4d74-81bd-f0005ef4f0c0"). InnerVolumeSpecName "kube-api-access-bhg67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:37.624971 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.624930 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhg67\" (UniqueName: \"kubernetes.io/projected/a1888a10-2980-4d74-81bd-f0005ef4f0c0-kube-api-access-bhg67\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:37.634048 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.634017 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:37.634411 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.634394 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" containerName="test-curl-7335-140017961499216" Jul 14 17:51:37.634493 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.634414 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" containerName="test-curl-7335-140017961499216" Jul 14 17:51:37.634493 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.634490 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" containerName="test-curl-7335-140017961499216" Jul 14 17:51:37.637368 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.637349 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:37.646226 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.646202 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:37.697812 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.697776 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1888a10-2980-4d74-81bd-f0005ef4f0c0" path="/var/lib/kubelet/pods/a1888a10-2980-4d74-81bd-f0005ef4f0c0/volumes" Jul 14 17:51:37.726034 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.726003 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vjzw\" (UniqueName: \"kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw\") pod \"test-curl-7335-140017961500224\" (UID: \"5f9f9446-0c23-4282-b893-23f2775d7d34\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:37.827211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.827172 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vjzw\" (UniqueName: \"kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw\") pod \"test-curl-7335-140017961500224\" (UID: \"5f9f9446-0c23-4282-b893-23f2775d7d34\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:37.841497 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.841464 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vjzw\" (UniqueName: \"kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw\") pod \"test-curl-7335-140017961500224\" (UID: \"5f9f9446-0c23-4282-b893-23f2775d7d34\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:37.947277 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:37.947192 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:38.086222 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.086191 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:38.087318 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:51:38.087289 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f9f9446_0c23_4282_b893_23f2775d7d34.slice/crio-f5cb6ac4d6bd926b1f05a91bc4807df5abf928953632b51934c98379e37afc36 WatchSource:0}: Error finding container f5cb6ac4d6bd926b1f05a91bc4807df5abf928953632b51934c98379e37afc36: Status 404 returned error can't find the container with id f5cb6ac4d6bd926b1f05a91bc4807df5abf928953632b51934c98379e37afc36 Jul 14 17:51:38.327878 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.327837 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961500224" event={"ID":"5f9f9446-0c23-4282-b893-23f2775d7d34","Type":"ContainerStarted","Data":"f5cb6ac4d6bd926b1f05a91bc4807df5abf928953632b51934c98379e37afc36"} Jul 14 17:51:38.329065 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.329046 2574 scope.go:117] "RemoveContainer" containerID="bef4dcc8cf3ebae6a969785ecfa8b8b8fb2525855c248ae90c30b552ef322160" Jul 14 17:51:38.329176 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.329068 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961499216" Jul 14 17:51:38.606004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.605898 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" podUID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" containerName="main" containerID="cri-o://367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46" gracePeriod=45 Jul 14 17:51:38.860042 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.859979 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:51:38.936424 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936384 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936447 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936476 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936547 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lmrz\" (UniqueName: \"kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936609 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936651 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936638 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs\") pod \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\" (UID: \"a5a0a060-e6a1-4e8c-9148-5c69d077d01e\") " Jul 14 17:51:38.936914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936749 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:51:38.936914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936838 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home" (OuterVolumeSpecName: "home") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:51:38.936914 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936848 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:51:38.937030 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936950 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-model-cache\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:38.937030 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936967 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tmp-dir\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:38.937030 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.936983 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-home\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:38.938866 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.938842 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz" (OuterVolumeSpecName: "kube-api-access-8lmrz") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "kube-api-access-8lmrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:38.938866 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.938855 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:51:38.938980 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:38.938840 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm" (OuterVolumeSpecName: "dshm") pod "a5a0a060-e6a1-4e8c-9148-5c69d077d01e" (UID: "a5a0a060-e6a1-4e8c-9148-5c69d077d01e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:51:39.037627 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.037557 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lmrz\" (UniqueName: \"kubernetes.io/projected/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-kube-api-access-8lmrz\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:39.037627 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.037626 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-dshm\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:39.037844 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.037638 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5a0a060-e6a1-4e8c-9148-5c69d077d01e-tls-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:39.337466 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.337434 2574 generic.go:356] "Generic (PLEG): container finished" podID="5f9f9446-0c23-4282-b893-23f2775d7d34" containerID="f20cb3246310d34e10af0ea1097f27f6f70ff5945ca65537ace7f035d3e8fd5d" exitCode=0 Jul 14 17:51:39.337669 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.337515 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961500224" event={"ID":"5f9f9446-0c23-4282-b893-23f2775d7d34","Type":"ContainerDied","Data":"f20cb3246310d34e10af0ea1097f27f6f70ff5945ca65537ace7f035d3e8fd5d"} Jul 14 17:51:39.338840 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.338813 2574 generic.go:356] "Generic (PLEG): container finished" podID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" containerID="367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46" exitCode=0 Jul 14 17:51:39.338955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.338843 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" Jul 14 17:51:39.338955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.338889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" event={"ID":"a5a0a060-e6a1-4e8c-9148-5c69d077d01e","Type":"ContainerDied","Data":"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46"} Jul 14 17:51:39.338955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.338924 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw" event={"ID":"a5a0a060-e6a1-4e8c-9148-5c69d077d01e","Type":"ContainerDied","Data":"4bda76bb83159ca3562b0efec7af532e321a835fafbc5e150cb8c411ce3f9fb1"} Jul 14 17:51:39.338955 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.338947 2574 scope.go:117] "RemoveContainer" containerID="367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46" Jul 14 17:51:39.348915 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.348892 2574 scope.go:117] "RemoveContainer" containerID="367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46" Jul 14 17:51:39.349228 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:51:39.349205 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46\": container with ID starting with 367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46 not found: ID does not exist" containerID="367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46" Jul 14 17:51:39.349324 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.349235 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46"} err="failed to get container status \"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46\": rpc error: code = NotFound desc = could not find container \"367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46\": container with ID starting with 367e9373420dfc1321c293400c93e6a6a7a73348bd9340e441f581be577bce46 not found: ID does not exist" Jul 14 17:51:39.371608 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.371551 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:51:39.377777 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.377747 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-246d80/rate-test-model-246d80-kserve-fbb8b8cc5-rq4gw"] Jul 14 17:51:39.692678 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:39.692595 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" path="/var/lib/kubelet/pods/a5a0a060-e6a1-4e8c-9148-5c69d077d01e/volumes" Jul 14 17:51:40.485487 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.485462 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:40.512517 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.512480 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961500224_5f9f9446-0c23-4282-b893-23f2775d7d34/test-curl-7335-140017961500224/0.log" Jul 14 17:51:40.550742 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.550711 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:40.558390 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.558364 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:40.650274 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.650181 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vjzw\" (UniqueName: \"kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw\") pod \"5f9f9446-0c23-4282-b893-23f2775d7d34\" (UID: \"5f9f9446-0c23-4282-b893-23f2775d7d34\") " Jul 14 17:51:40.652217 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.652184 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw" (OuterVolumeSpecName: "kube-api-access-7vjzw") pod "5f9f9446-0c23-4282-b893-23f2775d7d34" (UID: "5f9f9446-0c23-4282-b893-23f2775d7d34"). InnerVolumeSpecName "kube-api-access-7vjzw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:40.698506 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698472 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698790 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" containerName="main" Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698802 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" containerName="main" Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698826 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f9f9446-0c23-4282-b893-23f2775d7d34" containerName="test-curl-7335-140017961500224" Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698832 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f9f9446-0c23-4282-b893-23f2775d7d34" containerName="test-curl-7335-140017961500224" Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698882 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5a0a060-e6a1-4e8c-9148-5c69d077d01e" containerName="main" Jul 14 17:51:40.698896 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.698889 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f9f9446-0c23-4282-b893-23f2775d7d34" containerName="test-curl-7335-140017961500224" Jul 14 17:51:40.701818 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.701799 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:40.715029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.715001 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:40.751512 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.751481 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vjzw\" (UniqueName: \"kubernetes.io/projected/5f9f9446-0c23-4282-b893-23f2775d7d34-kube-api-access-7vjzw\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:40.851967 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.851927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkf7s\" (UniqueName: \"kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s\") pod \"test-curl-7335-140017961500224\" (UID: \"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:40.952752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.952662 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkf7s\" (UniqueName: \"kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s\") pod \"test-curl-7335-140017961500224\" (UID: \"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:40.963527 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:40.963494 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkf7s\" (UniqueName: \"kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s\") pod \"test-curl-7335-140017961500224\" (UID: \"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f\") " pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:41.010736 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.010697 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:41.154640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.154608 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:41.157454 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:51:41.157419 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf8ba6cd_9730_4bde_9ce3_edaeafe7a47f.slice/crio-b7722bfc0cb96135120eb255485c4112f83642a810d68bf127db67c2a5bccd49 WatchSource:0}: Error finding container b7722bfc0cb96135120eb255485c4112f83642a810d68bf127db67c2a5bccd49: Status 404 returned error can't find the container with id b7722bfc0cb96135120eb255485c4112f83642a810d68bf127db67c2a5bccd49 Jul 14 17:51:41.351768 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.351729 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961500224" event={"ID":"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f","Type":"ContainerStarted","Data":"b7722bfc0cb96135120eb255485c4112f83642a810d68bf127db67c2a5bccd49"} Jul 14 17:51:41.352867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.352841 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:41.352867 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.352865 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5cb6ac4d6bd926b1f05a91bc4807df5abf928953632b51934c98379e37afc36" Jul 14 17:51:41.365196 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.365169 2574 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7335-140017961500224" oldPodUID="5f9f9446-0c23-4282-b893-23f2775d7d34" podUID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" Jul 14 17:51:41.693029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:41.692942 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f9f9446-0c23-4282-b893-23f2775d7d34" path="/var/lib/kubelet/pods/5f9f9446-0c23-4282-b893-23f2775d7d34/volumes" Jul 14 17:51:42.360101 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:42.360068 2574 generic.go:356] "Generic (PLEG): container finished" podID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" containerID="13b9778b52cf4faf913ef91cd59a559a0ace924284b8d6bac4114c95d0fb926a" exitCode=0 Jul 14 17:51:42.360507 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:42.360149 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7335-140017961500224" event={"ID":"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f","Type":"ContainerDied","Data":"13b9778b52cf4faf913ef91cd59a559a0ace924284b8d6bac4114c95d0fb926a"} Jul 14 17:51:43.505758 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.505733 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:43.530881 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.530851 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7335-140017961500224_cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f/test-curl-7335-140017961500224/0.log" Jul 14 17:51:43.566638 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.566603 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:43.570698 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.570669 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7335-140017961500224"] Jul 14 17:51:43.676811 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.676701 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkf7s\" (UniqueName: \"kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s\") pod \"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f\" (UID: \"cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f\") " Jul 14 17:51:43.678925 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.678897 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s" (OuterVolumeSpecName: "kube-api-access-qkf7s") pod "cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" (UID: "cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f"). InnerVolumeSpecName "kube-api-access-qkf7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:51:43.693022 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.692994 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" path="/var/lib/kubelet/pods/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f/volumes" Jul 14 17:51:43.778036 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:43.778003 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkf7s\" (UniqueName: \"kubernetes.io/projected/cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f-kube-api-access-qkf7s\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:51:44.372142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:44.372103 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:44.372371 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:44.372325 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7335-140017961500224" Jul 14 17:51:44.372371 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:44.372361 2574 scope.go:117] "RemoveContainer" containerID="13b9778b52cf4faf913ef91cd59a559a0ace924284b8d6bac4114c95d0fb926a" Jul 14 17:51:44.472481 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:44.472447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:54.297802 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:54.297765 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:51:58.994972 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:51:58.994938 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:05.914069 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:05.914031 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:06.032981 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:06.032945 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:09.683883 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:09.683853 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:52:09.684268 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:09.684050 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" containerID="cri-o://6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b" gracePeriod=30 Jul 14 17:52:09.703980 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:09.703953 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:09.996389 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:09.996286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:11.019988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:11.019954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:11.115142 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:11.115110 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:14.520493 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:14.520459 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:14.621030 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:14.620996 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:14.969099 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:14.969075 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:52:15.056127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056034 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056105 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056127 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056129 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056147 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056174 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056202 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056237 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056266 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmrsp\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056421 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056298 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs\") pod \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\" (UID: \"90badcf8-5bf3-4fa8-bc4c-b11b2494b887\") " Jul 14 17:52:15.056688 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056509 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:52:15.056688 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056637 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.056789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.056716 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:52:15.057061 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.057027 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:52:15.057216 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.057112 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:52:15.057291 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.057257 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data" (OuterVolumeSpecName: "istio-data") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:52:15.058444 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.058410 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:52:15.058541 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.058482 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token" (OuterVolumeSpecName: "istio-token") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:52:15.058680 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.058661 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:52:15.058806 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.058790 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp" (OuterVolumeSpecName: "kube-api-access-lmrsp") pod "90badcf8-5bf3-4fa8-bc4c-b11b2494b887" (UID: "90badcf8-5bf3-4fa8-bc4c-b11b2494b887"). InnerVolumeSpecName "kube-api-access-lmrsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:52:15.158085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158038 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-data\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158076 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmrsp\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-kube-api-access-lmrsp\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158085 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158090 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-workload-certs\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158105 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-token\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158116 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istiod-ca-cert\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158127 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-podinfo\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158137 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-istio-envoy\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.158339 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.158148 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90badcf8-5bf3-4fa8-bc4c-b11b2494b887-credential-socket\") on node \"ip-10-0-129-213.ec2.internal\" DevicePath \"\"" Jul 14 17:52:15.530762 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.530728 2574 generic.go:356] "Generic (PLEG): container finished" podID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerID="6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b" exitCode=0 Jul 14 17:52:15.531174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.530772 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" Jul 14 17:52:15.531174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.530820 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerDied","Data":"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b"} Jul 14 17:52:15.531174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.530879 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj" event={"ID":"90badcf8-5bf3-4fa8-bc4c-b11b2494b887","Type":"ContainerDied","Data":"85f836c933ee62a695de30b9695b7fd64f9e0c639500fbf0dd55f08d704cea0b"} Jul 14 17:52:15.531174 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.530909 2574 scope.go:117] "RemoveContainer" containerID="6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b" Jul 14 17:52:15.540252 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.540235 2574 scope.go:117] "RemoveContainer" containerID="6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e" Jul 14 17:52:15.548403 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.548381 2574 scope.go:117] "RemoveContainer" containerID="6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b" Jul 14 17:52:15.548660 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:52:15.548641 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b\": container with ID starting with 6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b not found: ID does not exist" containerID="6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b" Jul 14 17:52:15.548724 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.548669 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b"} err="failed to get container status \"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b\": rpc error: code = NotFound desc = could not find container \"6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b\": container with ID starting with 6eb154b78cd94b3cba9ce6da5de62ffb16997fd5ffb90e70f6d79f2b3e28980b not found: ID does not exist" Jul 14 17:52:15.548724 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.548687 2574 scope.go:117] "RemoveContainer" containerID="6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e" Jul 14 17:52:15.548938 ip-10-0-129-213 kubenswrapper[2574]: E0714 17:52:15.548921 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e\": container with ID starting with 6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e not found: ID does not exist" containerID="6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e" Jul 14 17:52:15.548986 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.548943 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e"} err="failed to get container status \"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e\": rpc error: code = NotFound desc = could not find container \"6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e\": container with ID starting with 6d37635023e3e942952111f2ff9853f917d9003f1317121fc9e4e1a3f98cff9e not found: ID does not exist" Jul 14 17:52:15.555369 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.555345 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:52:15.560289 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.560264 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-246d80-openshift-default-5ddbb88bdc-cvdvj"] Jul 14 17:52:15.693221 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:15.693187 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" path="/var/lib/kubelet/pods/90badcf8-5bf3-4fa8-bc4c-b11b2494b887/volumes" Jul 14 17:52:20.780257 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:20.780222 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:25.243038 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:25.243005 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:31.949665 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:31.949629 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:32.024164 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:32.024130 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:35.963004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:35.962972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:36.194396 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:36.194358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:36.605988 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:36.605958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c47c5648d-frff2_58e096c0-2a88-4c81-839e-fc0447eddf6b/authorino/0.log" Jul 14 17:52:41.973953 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:41.973919 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64846ff95c-nd78g_a1e6b131-607c-4fe5-b265-22333a5c5204/manager/0.log" Jul 14 17:52:42.486723 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:42.486692 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-w8tdf_3cfb5224-e6bc-453b-8f47-e0a14a13900e/copy-manifests/0.log" Jul 14 17:52:42.500456 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:42.500417 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-w8tdf_3cfb5224-e6bc-453b-8f47-e0a14a13900e/manager/0.log" Jul 14 17:52:45.643047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.643007 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/extract/0.log" Jul 14 17:52:45.657079 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.657053 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/util/0.log" Jul 14 17:52:45.665631 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.665605 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/pull/0.log" Jul 14 17:52:45.779865 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.779837 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/util/0.log" Jul 14 17:52:45.788769 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.788727 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/pull/0.log" Jul 14 17:52:45.798725 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.798694 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/extract/0.log" Jul 14 17:52:45.911781 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.911689 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/pull/0.log" Jul 14 17:52:45.920027 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.919997 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/extract/0.log" Jul 14 17:52:45.927629 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:45.927606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/util/0.log" Jul 14 17:52:46.051327 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:46.051297 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/util/0.log" Jul 14 17:52:46.068943 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:46.068915 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/pull/0.log" Jul 14 17:52:46.087463 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:46.087432 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/extract/0.log" Jul 14 17:52:46.199823 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:46.199729 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c47c5648d-frff2_58e096c0-2a88-4c81-839e-fc0447eddf6b/authorino/0.log" Jul 14 17:52:47.416109 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:47.416075 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zmsd9_c5f7dfcd-8c5e-4eab-8ced-e2f172da05be/istio-proxy/0.log" Jul 14 17:52:47.777374 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:47.777292 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-frzt5_a2bfe160-ae19-4df2-90d2-de4e7a8e28ab/kube-auth-proxy/0.log" Jul 14 17:52:48.735654 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:48.735620 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qqqt7_26c3f1f9-5c97-4278-ae0c-6df0611f89c0/main/0.log" Jul 14 17:52:48.971727 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:48.971698 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6h6zv_19cbca68-e09b-4238-991b-cf74068239b6/main/0.log" Jul 14 17:52:49.197980 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:49.197951 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6z5w7_0b217724-f3e8-4699-bae5-fa22a6880fbe/main/0.log" Jul 14 17:52:53.302559 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302528 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bv25z/must-gather-gxkzm"] Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302867 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302878 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302891 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" containerName="test-curl-7335-140017961500224" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302898 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" containerName="test-curl-7335-140017961500224" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302905 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302912 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302960 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.302961 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302967 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="90badcf8-5bf3-4fa8-bc4c-b11b2494b887" containerName="istio-proxy" Jul 14 17:52:53.303222 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.302977 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf8ba6cd-9730-4bde-9ce3-edaeafe7a47f" containerName="test-curl-7335-140017961500224" Jul 14 17:52:53.305945 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.305929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.309047 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.309018 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"kube-root-ca.crt\"" Jul 14 17:52:53.312097 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.312052 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bv25z\"/\"default-dockercfg-lld7c\"" Jul 14 17:52:53.312340 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.312046 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bv25z\"/\"openshift-service-ca.crt\"" Jul 14 17:52:53.322357 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.322334 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/must-gather-gxkzm"] Jul 14 17:52:53.329951 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.329892 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:53.431294 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.431258 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" limit=1024 Jul 14 17:52:53.479308 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.479267 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb0814cc-aa04-4a0a-a92c-173ab7325c58-must-gather-output\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.479308 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.479313 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv57n\" (UniqueName: \"kubernetes.io/projected/eb0814cc-aa04-4a0a-a92c-173ab7325c58-kube-api-access-mv57n\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.580237 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.580208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv57n\" (UniqueName: \"kubernetes.io/projected/eb0814cc-aa04-4a0a-a92c-173ab7325c58-kube-api-access-mv57n\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.580439 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.580296 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb0814cc-aa04-4a0a-a92c-173ab7325c58-must-gather-output\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.580647 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.580609 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eb0814cc-aa04-4a0a-a92c-173ab7325c58-must-gather-output\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.592637 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.592605 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv57n\" (UniqueName: \"kubernetes.io/projected/eb0814cc-aa04-4a0a-a92c-173ab7325c58-kube-api-access-mv57n\") pod \"must-gather-gxkzm\" (UID: \"eb0814cc-aa04-4a0a-a92c-173ab7325c58\") " pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.616569 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.616541 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/must-gather-gxkzm" Jul 14 17:52:53.751828 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:53.751799 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/must-gather-gxkzm"] Jul 14 17:52:53.753728 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:52:53.753701 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb0814cc_aa04_4a0a_a92c_173ab7325c58.slice/crio-812361c72f60ff7430395a740c40d05580216de17ee0c88fae27c67e325a9057 WatchSource:0}: Error finding container 812361c72f60ff7430395a740c40d05580216de17ee0c88fae27c67e325a9057: Status 404 returned error can't find the container with id 812361c72f60ff7430395a740c40d05580216de17ee0c88fae27c67e325a9057 Jul 14 17:52:54.728330 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:54.728290 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/must-gather-gxkzm" event={"ID":"eb0814cc-aa04-4a0a-a92c-173ab7325c58","Type":"ContainerStarted","Data":"c6c46617ad2ae83c14a6a6456207f2a72007c3c85587c7c98c008f652760bbe1"} Jul 14 17:52:54.728787 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:54.728344 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/must-gather-gxkzm" event={"ID":"eb0814cc-aa04-4a0a-a92c-173ab7325c58","Type":"ContainerStarted","Data":"812361c72f60ff7430395a740c40d05580216de17ee0c88fae27c67e325a9057"} Jul 14 17:52:55.738546 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:55.738499 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/must-gather-gxkzm" event={"ID":"eb0814cc-aa04-4a0a-a92c-173ab7325c58","Type":"ContainerStarted","Data":"60165d96fcf31248347d3f4a873bae89ec22ee2a2f696dd6923007623905c366"} Jul 14 17:52:56.222837 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:56.222809 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-46tkm_dba23291-95f1-4b81-9c35-80a0955c5935/global-pull-secret-syncer/0.log" Jul 14 17:52:56.395360 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:56.395331 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-klggv_70960940-3298-4dee-842a-61b2d4494306/konnectivity-agent/0.log" Jul 14 17:52:56.542909 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:52:56.542834 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-213.ec2.internal_d98126d381549eedd58b496d19efe54c/haproxy/0.log" Jul 14 17:53:01.248320 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.248224 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/extract/0.log" Jul 14 17:53:01.292120 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.292080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/util/0.log" Jul 14 17:53:01.320326 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.320284 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gmwzf_ff7519d1-65c8-4936-b255-a4def886833c/pull/0.log" Jul 14 17:53:01.365457 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.365427 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/extract/0.log" Jul 14 17:53:01.399310 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.399274 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/util/0.log" Jul 14 17:53:01.430382 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.430346 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08w8qk_b8bbfd0a-e37e-481b-879e-d8b898ad5599/pull/0.log" Jul 14 17:53:01.473441 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.473401 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/extract/0.log" Jul 14 17:53:01.509298 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.509215 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/util/0.log" Jul 14 17:53:01.537713 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.537676 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73srn24_578685c7-a6d8-401d-9bc8-d45ebed7d344/pull/0.log" Jul 14 17:53:01.576132 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.576097 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/extract/0.log" Jul 14 17:53:01.607895 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.607858 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/util/0.log" Jul 14 17:53:01.640029 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:01.639778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q84w9_2d5e81c1-2550-4aaa-a881-e146827f07c0/pull/0.log" Jul 14 17:53:02.077295 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:02.077262 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c47c5648d-frff2_58e096c0-2a88-4c81-839e-fc0447eddf6b/authorino/0.log" Jul 14 17:53:04.664056 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:04.664020 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkwzc_3a860158-6ffe-4345-8c2a-e1661cb29d76/node-exporter/0.log" Jul 14 17:53:04.690408 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:04.690373 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkwzc_3a860158-6ffe-4345-8c2a-e1661cb29d76/kube-rbac-proxy/0.log" Jul 14 17:53:04.714770 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:04.714739 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-dkwzc_3a860158-6ffe-4345-8c2a-e1661cb29d76/init-textfile/0.log" Jul 14 17:53:06.090515 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.090447 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bv25z/must-gather-gxkzm" podStartSLOduration=12.278152408 podStartE2EDuration="13.090423759s" totalImagesPullingTime="812.271351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:52:55.761161546 +0000 UTC m=+2869.284102320" watchObservedRunningTime="2026-07-14 17:53:06.090423759 +0000 UTC m=+2879.613364531" Jul 14 17:53:06.091032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.090679 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh"] Jul 14 17:53:06.095679 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.095648 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.105789 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.105754 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh"] Jul 14 17:53:06.197702 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.197665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-proc\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.197897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.197733 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-sys\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.197897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.197794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-podres\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.197897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.197826 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-lib-modules\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.197897 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.197847 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29jcd\" (UniqueName: \"kubernetes.io/projected/5219bc07-df92-4d8a-a7ae-2a224791a776-kube-api-access-29jcd\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299113 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-podres\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299140 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-lib-modules\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29jcd\" (UniqueName: \"kubernetes.io/projected/5219bc07-df92-4d8a-a7ae-2a224791a776-kube-api-access-29jcd\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-proc\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-podres\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299313 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299286 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-sys\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299528 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299318 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-lib-modules\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299528 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299354 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-proc\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.299528 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.299381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5219bc07-df92-4d8a-a7ae-2a224791a776-sys\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.309004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.308974 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29jcd\" (UniqueName: \"kubernetes.io/projected/5219bc07-df92-4d8a-a7ae-2a224791a776-kube-api-access-29jcd\") pod \"perf-node-gather-daemonset-42vfh\" (UID: \"5219bc07-df92-4d8a-a7ae-2a224791a776\") " pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.410338 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.410231 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.595756 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.595712 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh"] Jul 14 17:53:06.597434 ip-10-0-129-213 kubenswrapper[2574]: W0714 17:53:06.597354 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5219bc07_df92_4d8a_a7ae_2a224791a776.slice/crio-7ca401f6d995d0c5deb7932310e6a4cad0ab19ec825ebcbecc2b3f45deef8f2b WatchSource:0}: Error finding container 7ca401f6d995d0c5deb7932310e6a4cad0ab19ec825ebcbecc2b3f45deef8f2b: Status 404 returned error can't find the container with id 7ca401f6d995d0c5deb7932310e6a4cad0ab19ec825ebcbecc2b3f45deef8f2b Jul 14 17:53:06.829985 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.829941 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" event={"ID":"5219bc07-df92-4d8a-a7ae-2a224791a776","Type":"ContainerStarted","Data":"307f9e9b219fe88c33d86e8e9697be8e8f420c61ec34cbeaaf5da04b365caf1a"} Jul 14 17:53:06.830190 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.830018 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" event={"ID":"5219bc07-df92-4d8a-a7ae-2a224791a776","Type":"ContainerStarted","Data":"7ca401f6d995d0c5deb7932310e6a4cad0ab19ec825ebcbecc2b3f45deef8f2b"} Jul 14 17:53:06.830609 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.830566 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:06.857629 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:06.857550 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" podStartSLOduration=0.857532049 podStartE2EDuration="857.532049ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:53:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:53:06.856673605 +0000 UTC m=+2880.379614379" watchObservedRunningTime="2026-07-14 17:53:06.857532049 +0000 UTC m=+2880.380472822" Jul 14 17:53:07.938049 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:07.938017 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-lp9p6_742b8fa1-9d7e-4e17-8fb1-5f211689026d/download-server/0.log" Jul 14 17:53:09.783004 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:09.782969 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kg8cw_4d8fd9ff-d18b-46e4-a3d5-3da9c997b619/dns/0.log" Jul 14 17:53:09.812529 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:09.812502 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kg8cw_4d8fd9ff-d18b-46e4-a3d5-3da9c997b619/kube-rbac-proxy/0.log" Jul 14 17:53:09.868809 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:09.868781 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-csj2m_88367bbb-0442-4660-9d3f-b7cc31e51be1/dns-node-resolver/0.log" Jul 14 17:53:10.450153 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:10.450118 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8476656947-58b5p_d3658f22-9430-4dd4-b020-52dd9f408473/registry/0.log" Jul 14 17:53:10.561211 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:10.561142 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4sw8m_425266e1-079d-46b8-8366-ce4b5651f269/node-ca/0.log" Jul 14 17:53:11.557757 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:11.557722 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zmsd9_c5f7dfcd-8c5e-4eab-8ced-e2f172da05be/istio-proxy/0.log" Jul 14 17:53:11.826093 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:11.826007 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-frzt5_a2bfe160-ae19-4df2-90d2-de4e7a8e28ab/kube-auth-proxy/0.log" Jul 14 17:53:12.802918 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:12.802884 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x9nmv_24fa006d-f6dc-4f46-83b1-cee1d2e66972/serve-healthcheck-canary/0.log" Jul 14 17:53:12.849929 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:12.849886 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bv25z/perf-node-gather-daemonset-42vfh" Jul 14 17:53:13.531236 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:13.531203 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkwln_e11e13b8-944b-47ed-8c06-1f8082cc3830/kube-rbac-proxy/0.log" Jul 14 17:53:13.554551 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:13.554526 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkwln_e11e13b8-944b-47ed-8c06-1f8082cc3830/exporter/0.log" Jul 14 17:53:13.578963 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:13.578935 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jkwln_e11e13b8-944b-47ed-8c06-1f8082cc3830/extractor/0.log" Jul 14 17:53:16.512587 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:16.512488 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-64846ff95c-nd78g_a1e6b131-607c-4fe5-b265-22333a5c5204/manager/0.log" Jul 14 17:53:16.706752 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:16.706720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zmsd9_c5f7dfcd-8c5e-4eab-8ced-e2f172da05be/istio-proxy/0.log" Jul 14 17:53:16.803091 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:16.803014 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-w8tdf_3cfb5224-e6bc-453b-8f47-e0a14a13900e/manager/0.log" Jul 14 17:53:16.823510 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:16.823485 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7655f47d9d-w8tdf_3cfb5224-e6bc-453b-8f47-e0a14a13900e/copy-manifests/0.log" Jul 14 17:53:16.935640 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:16.935606 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-frzt5_a2bfe160-ae19-4df2-90d2-de4e7a8e28ab/kube-auth-proxy/0.log" Jul 14 17:53:18.250894 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:18.250819 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5684c55548-djz47_29fcdfcb-3e96-4afa-a416-99fcafef228c/manager/0.log" Jul 14 17:53:18.282209 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:18.282178 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6ldrv_ca935859-3ba3-4928-9cf0-74d475cf323d/openshift-lws-operator/0.log" Jul 14 17:53:18.467138 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:18.467099 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4zmsd9_c5f7dfcd-8c5e-4eab-8ced-e2f172da05be/istio-proxy/0.log" Jul 14 17:53:18.722080 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:18.722056 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c548b7c75-frzt5_a2bfe160-ae19-4df2-90d2-de4e7a8e28ab/kube-auth-proxy/0.log" Jul 14 17:53:23.322032 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:23.321938 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-g9z2z_32bb29f2-1a48-4524-b6d8-3fd711ee6fb3/migrator/0.log" Jul 14 17:53:23.360349 ip-10-0-129-213 kubenswrapper[2574]: I0714 17:53:23.360305 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-g9z2z_32bb29f2-1a48-4524-b6d8-3fd711ee6fb3/graceful-termination/0.log"