Jul 22 11:21:40.821560 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 11:21:40.821575 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 11:21:40.821585 ip-10-0-135-12 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 11:21:40.821910 ip-10-0-135-12 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 11:21:50.910105 ip-10-0-135-12 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 11:21:50.910122 ip-10-0-135-12 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 35284cf635c34436aea53572f28d61a8 -- Jul 22 11:23:49.566718 ip-10-0-135-12 systemd[1]: Starting Kubernetes Kubelet... Jul 22 11:23:49.991903 ip-10-0-135-12 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:23:49.991903 ip-10-0-135-12 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 11:23:49.991903 ip-10-0-135-12 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:23:49.991903 ip-10-0-135-12 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 11:23:49.991903 ip-10-0-135-12 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 11:23:49.994564 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:49.994428 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 11:23:49.998591 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998575 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:49.998591 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998591 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998595 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998598 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998601 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998603 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998608 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998611 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998614 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998616 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998619 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998622 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998625 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998628 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998630 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998634 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998638 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998641 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998644 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998646 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:49.998656 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998648 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998651 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998653 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998656 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998658 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998661 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998664 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998666 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998670 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998672 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998675 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998677 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998680 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998682 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998687 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998691 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998693 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998695 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998699 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:49.999192 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998703 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998705 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998708 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998711 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998713 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998716 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998718 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998721 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998723 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998725 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998728 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998730 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998732 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998734 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998737 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998740 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998743 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998745 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998748 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:49.999703 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998751 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998753 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998756 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998758 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998761 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998763 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998766 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998768 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998771 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998773 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998775 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998778 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998780 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998782 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998785 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998787 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998790 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998792 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998794 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998797 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.000288 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998800 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998802 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998804 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998806 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998809 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998812 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998815 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998818 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998822 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998824 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998828 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998830 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998833 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998835 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998837 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998840 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998843 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998845 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998848 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998851 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.000976 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.998853 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999556 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999563 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999567 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999569 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999572 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999574 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999579 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999583 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999587 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999590 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999592 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999596 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999599 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999602 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999604 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999607 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999609 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999612 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:50.001482 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999615 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999617 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999620 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999622 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999625 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999628 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999630 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999633 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999635 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999638 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999641 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999645 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999648 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999652 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999655 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999658 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999660 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999662 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999665 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999668 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.001958 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999670 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999674 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999677 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999680 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999683 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999686 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999689 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999691 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999694 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999696 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999699 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999702 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999705 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999707 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999710 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999712 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999715 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999717 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999720 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:50.002470 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999722 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999725 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999727 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999729 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999732 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999734 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999737 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999740 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999742 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999744 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999747 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999749 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999752 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999754 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999758 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999760 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999762 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999765 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999767 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999769 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.002932 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999772 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999775 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999777 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999779 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999782 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999785 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:49.999787 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000113 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000125 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000131 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000136 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000141 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000145 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000150 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000155 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000160 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000164 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000169 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000177 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.003425 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000188 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000193 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.000197 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001172 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001182 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001188 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001191 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001194 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001196 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001199 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001202 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001205 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001207 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001210 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001213 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001215 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001218 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001220 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001223 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:50.003892 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001226 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001229 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001232 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001234 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001237 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001239 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001242 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001244 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001246 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001249 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001251 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001254 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001259 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001262 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001264 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001267 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001269 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001272 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001275 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001277 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:50.004386 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001279 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001282 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001285 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001287 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001290 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001292 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001295 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001298 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001300 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001303 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001305 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001310 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001315 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001319 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001322 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001324 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001327 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001329 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001332 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.004878 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001335 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001337 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001340 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001342 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001345 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001348 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001351 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001354 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001356 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001359 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001361 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001363 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001367 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001370 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001372 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001375 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001379 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001381 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001383 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001386 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:50.005367 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001388 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001391 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001393 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001396 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001399 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001401 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001404 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001406 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001410 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001414 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001417 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001420 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001423 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001426 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001428 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001448 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001450 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001454 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001457 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001460 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.005884 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001463 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001465 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001468 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.001471 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001547 2577 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001555 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001564 2577 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001571 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001578 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001583 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001587 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001591 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001594 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001597 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001600 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001603 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001607 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001610 2577 flags.go:64] FLAG: --cgroup-root="" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001613 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001616 2577 flags.go:64] FLAG: --client-ca-file="" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001618 2577 flags.go:64] FLAG: --cloud-provider="external" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001621 2577 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001626 2577 flags.go:64] FLAG: --cluster-domain="" Jul 22 11:23:50.006369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001629 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001632 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001634 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001637 2577 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001641 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001644 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001647 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001650 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001653 2577 flags.go:64] FLAG: --contention-profiling="false" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001656 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001659 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001661 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001664 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001668 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001671 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001674 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001677 2577 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001679 2577 flags.go:64] FLAG: --enable-server="true" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001682 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001686 2577 flags.go:64] FLAG: --event-burst="100" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001689 2577 flags.go:64] FLAG: --event-qps="50" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001692 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001694 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001697 2577 flags.go:64] FLAG: --eviction-hard="" Jul 22 11:23:50.006975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001700 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001704 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001707 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001710 2577 flags.go:64] FLAG: --eviction-soft="" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001713 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001715 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001718 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001721 2577 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001725 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001728 2577 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001730 2577 flags.go:64] FLAG: --feature-gates="" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001733 2577 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001736 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001739 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001742 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001745 2577 flags.go:64] FLAG: --healthz-port="10248" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001748 2577 flags.go:64] FLAG: --help="false" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001750 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001754 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001757 2577 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001759 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001762 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001765 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001768 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 11:23:50.007549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001771 2577 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001774 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001777 2577 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001780 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001791 2577 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001794 2577 flags.go:64] FLAG: --kube-reserved="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001796 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001799 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001802 2577 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001805 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001808 2577 flags.go:64] FLAG: --lock-file="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001811 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001813 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001816 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001821 2577 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001824 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001826 2577 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001829 2577 flags.go:64] FLAG: --logging-format="text" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001832 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001835 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001838 2577 flags.go:64] FLAG: --manifest-url="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001840 2577 flags.go:64] FLAG: --manifest-url-header="" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001845 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001848 2577 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001852 2577 flags.go:64] FLAG: --max-pods="110" Jul 22 11:23:50.008174 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001854 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001857 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001860 2577 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001862 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001865 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001868 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001871 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001878 2577 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001882 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001885 2577 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001888 2577 flags.go:64] FLAG: --pod-cidr="" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001891 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001897 2577 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001899 2577 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001902 2577 flags.go:64] FLAG: --pods-per-core="0" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001905 2577 flags.go:64] FLAG: --port="10250" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001908 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001911 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d75a7bda0f5aa64d" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001914 2577 flags.go:64] FLAG: --qos-reserved="" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001917 2577 flags.go:64] FLAG: --read-only-port="10255" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001919 2577 flags.go:64] FLAG: --register-node="true" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001922 2577 flags.go:64] FLAG: --register-with-taints="" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001925 2577 flags.go:64] FLAG: --registry-burst="10" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001928 2577 flags.go:64] FLAG: --registry-qps="5" Jul 22 11:23:50.008834 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001931 2577 flags.go:64] FLAG: --reserved-cpus="" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001933 2577 flags.go:64] FLAG: --reserved-memory="" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001937 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001940 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001942 2577 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001945 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001947 2577 flags.go:64] FLAG: --runonce="false" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001950 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001953 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001955 2577 flags.go:64] FLAG: --seccomp-default="false" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001958 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001961 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001964 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001967 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001971 2577 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001974 2577 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001976 2577 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001980 2577 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001983 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001985 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001988 2577 flags.go:64] FLAG: --system-cgroups="" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001991 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001996 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.001998 2577 flags.go:64] FLAG: --tls-cert-file="" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002004 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 11:23:50.009424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002008 2577 flags.go:64] FLAG: --tls-min-version="" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002013 2577 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002015 2577 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002018 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002021 2577 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002024 2577 flags.go:64] FLAG: --v="2" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002027 2577 flags.go:64] FLAG: --version="false" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002031 2577 flags.go:64] FLAG: --vmodule="" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002036 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.002039 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002181 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002186 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002190 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002194 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002196 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002199 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002202 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002205 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002207 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002210 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002213 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002215 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.010059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002218 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002220 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002223 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002226 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002229 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002231 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002234 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002236 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002239 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002243 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002246 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002250 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002252 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002255 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002257 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002259 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002262 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002264 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002267 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002269 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.010663 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002272 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002274 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002277 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002280 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002282 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002285 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002287 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002290 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002292 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002294 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002297 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002299 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002302 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002304 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002307 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002309 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002313 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002315 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002318 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002320 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:50.011159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002323 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002327 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002330 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002333 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002336 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002338 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002341 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002343 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002346 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002348 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002350 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002353 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002355 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002358 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002360 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002362 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002366 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002370 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002374 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.011659 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002377 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002379 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002382 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002385 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002387 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002390 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002393 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002395 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002398 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002400 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002403 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002405 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002408 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002410 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002414 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002417 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002421 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002423 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002426 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002444 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:50.012116 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002447 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002450 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002452 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002455 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002458 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002460 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002463 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.002465 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.003013 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.009926 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.009943 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.009963 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.009972 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010027 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010032 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:50.012678 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010035 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010038 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010041 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010044 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010046 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010049 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010051 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010054 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010056 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010059 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010061 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010066 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010070 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010074 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010077 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010080 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010082 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010085 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010088 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010091 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:50.013052 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010094 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010097 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010099 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010102 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010105 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010107 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010111 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010115 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010118 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010120 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010123 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010126 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010128 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010131 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010134 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010136 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010139 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010141 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010143 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010146 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:50.013560 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010148 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010151 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010154 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010156 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010159 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010161 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010164 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010166 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010168 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010171 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010173 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010177 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010181 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010183 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010186 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010188 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010191 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010193 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010195 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.014064 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010199 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010202 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010205 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010208 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010210 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010212 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010216 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010219 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010222 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010225 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010228 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010231 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010234 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010236 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010239 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010241 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010244 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010246 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010249 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:50.014554 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010251 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010254 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010256 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010259 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010261 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010264 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010266 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010269 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010271 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010273 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010276 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010278 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010281 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010284 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010287 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010290 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010293 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010295 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:50.015010 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010297 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.010302 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010420 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010424 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010427 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010450 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010455 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010458 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010460 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010463 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010465 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010468 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010470 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010473 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010475 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 11:23:50.015461 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010478 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010480 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010483 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010486 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010489 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010491 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010493 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010496 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010498 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010501 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010503 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010506 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010508 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010511 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010514 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010517 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010520 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010522 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010525 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010527 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 11:23:50.015832 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010529 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010533 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010537 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010540 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010542 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010545 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010547 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010550 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010552 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010554 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010557 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010559 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010561 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010564 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010566 2577 feature_gate.go:328] unrecognized feature gate: Example Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010569 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010571 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010574 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010576 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 11:23:50.016400 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010579 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010581 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010584 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010586 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010589 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010591 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010594 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010596 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010598 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010603 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010606 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010609 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010612 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010614 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010617 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010620 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010622 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010625 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010627 2577 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 11:23:50.016869 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010629 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010632 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010634 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010636 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010639 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010641 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010644 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010646 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010649 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010651 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010654 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010656 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010659 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010661 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010663 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010666 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010668 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010671 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010675 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 11:23:50.017326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010679 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010681 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010684 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010686 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010689 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010692 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010694 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010696 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:50.010699 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.010704 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.010814 2577 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.013551 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.014714 2577 server.go:1019] "Starting client certificate rotation" Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.014815 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 11:23:50.017806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.014852 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 11:23:50.018239 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.018226 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 11:23:50.021830 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.021810 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 11:23:50.032124 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.032101 2577 log.go:25] "Validated CRI v1 runtime API" Jul 22 11:23:50.034218 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.034196 2577 log.go:25] "Validated CRI v1 image API" Jul 22 11:23:50.037063 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.037046 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 11:23:50.040179 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.040159 2577 fs.go:135] Filesystem UUIDs: map[03e91874-11af-4335-bcef-d3c411f2ebb9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ed3e1478-3664-4dd1-97f7-a656f2eb6f42:/dev/nvme0n1p3] Jul 22 11:23:50.040240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.040179 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 11:23:50.045821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.045710 2577 manager.go:216] Machine: {Timestamp:2026-07-22 11:23:50.044513387 +0000 UTC m=+0.359575675 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100362 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23d6baaf09d626017fccbe0617aab4 SystemUUID:ec23d6ba-af09-d626-017f-ccbe0617aab4 BootID:35284cf6-35c3-4436-aea5-3572f28d61a8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5f:73:d8:7d:01 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5f:73:d8:7d:01 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:12:2c:42:cf:e3:1d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 11:23:50.045821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.045817 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 11:23:50.045934 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.045894 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 11:23:50.046329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.046314 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 11:23:50.046859 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.046832 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 11:23:50.046993 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.046862 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 11:23:50.048068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.048058 2577 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 11:23:50.048105 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.048071 2577 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 11:23:50.048105 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.048078 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 11:23:50.048757 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.048747 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 11:23:50.051069 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.051059 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 11:23:50.053700 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053689 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 22 11:23:50.053871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053861 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 11:23:50.053908 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053891 2577 kubelet.go:486] "Attempting to sync node with API server" Jul 22 11:23:50.053908 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053905 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 11:23:50.053967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053922 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 11:23:50.053967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053938 2577 kubelet.go:398] "Adding apiserver pod source" Jul 22 11:23:50.053967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.053948 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 11:23:50.054308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.054295 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 11:23:50.054356 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.054314 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 11:23:50.056739 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.056724 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 11:23:50.058466 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.058451 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 11:23:50.058523 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.058481 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 11:23:50.059465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059428 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 11:23:50.059465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059460 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 11:23:50.059465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059466 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059474 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059482 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059488 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059496 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059501 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059507 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059512 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059526 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 11:23:50.059611 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.059531 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 11:23:50.060257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.060247 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 11:23:50.060257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.060255 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 11:23:50.063818 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.063802 2577 server.go:1268] "Started kubelet" Jul 22 11:23:50.063902 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.063863 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 11:23:50.063902 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.063870 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 11:23:50.064014 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.063958 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 11:23:50.064740 ip-10-0-135-12 systemd[1]: Started Kubernetes Kubelet. Jul 22 11:23:50.065471 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.065451 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 11:23:50.070503 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.070469 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 11:23:50.070603 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.070507 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 11:23:50.070603 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.070531 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 11:23:50.070774 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.070740 2577 server.go:310] "Adding debug handlers to kubelet server" Jul 22 11:23:50.070875 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.070854 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 11:23:50.073859 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.073725 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 11:23:50.073859 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.072397 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-12.ec2.internal.18c498dda4b0623c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-12.ec2.internal,UID:ip-10-0-135-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-12.ec2.internal,},FirstTimestamp:2026-07-22 11:23:50.063776316 +0000 UTC m=+0.378838603,LastTimestamp:2026-07-22 11:23:50.063776316 +0000 UTC m=+0.378838603,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-12.ec2.internal,}" Jul 22 11:23:50.074085 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.073897 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.074231 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.074712 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tftxc" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.074916 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.075020 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.075037 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.075199 2577 reconstruct.go:97] "Volume reconstruction finished" Jul 22 11:23:50.075806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.075208 2577 reconciler.go:29] "Reconciler: start to sync state" Jul 22 11:23:50.076152 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076142 2577 factory.go:55] Registering systemd factory Jul 22 11:23:50.076194 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076158 2577 factory.go:223] Registration of the systemd container factory successfully Jul 22 11:23:50.076693 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076671 2577 factory.go:153] Registering CRI-O factory Jul 22 11:23:50.076776 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076713 2577 factory.go:223] Registration of the crio container factory successfully Jul 22 11:23:50.076832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076783 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 11:23:50.076832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076805 2577 factory.go:103] Registering Raw factory Jul 22 11:23:50.076832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.076822 2577 manager.go:1196] Started watching for new ooms in manager Jul 22 11:23:50.077500 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.077484 2577 manager.go:349] Starting recovery of all containers Jul 22 11:23:50.078295 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.078209 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 11:23:50.078418 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.078338 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 11:23:50.087050 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.086895 2577 manager.go:354] Recovery completed Jul 22 11:23:50.087162 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.087143 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tftxc" Jul 22 11:23:50.091232 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.091217 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.093961 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.093947 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.094019 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.093972 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.094019 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.093982 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.094575 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.094562 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 11:23:50.094575 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.094574 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 11:23:50.094671 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.094590 2577 state_mem.go:36] "Initialized new in-memory state store" Jul 22 11:23:50.094671 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.094568 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.095338 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095322 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.095379 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095361 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.095379 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095372 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.095686 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095674 2577 policy_none.go:49] "None policy: Start" Jul 22 11:23:50.095729 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095691 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 11:23:50.095729 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.095701 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 11:23:50.096055 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.095996 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-12.ec2.internal.18c498dda67cf359 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-12.ec2.internal,UID:ip-10-0-135-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-12.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-12.ec2.internal,},FirstTimestamp:2026-07-22 11:23:50.093960025 +0000 UTC m=+0.409022314,LastTimestamp:2026-07-22 11:23:50.093960025 +0000 UTC m=+0.409022314,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-12.ec2.internal,}" Jul 22 11:23:50.096743 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.096732 2577 policy_none.go:47] "Start" Jul 22 11:23:50.134045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134027 2577 manager.go:337] "Starting Device Plugin manager" Jul 22 11:23:50.134127 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.134065 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 11:23:50.134127 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134078 2577 server.go:87] "Starting device plugin registration server" Jul 22 11:23:50.134344 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134331 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 11:23:50.134385 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134347 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 11:23:50.134492 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134468 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 11:23:50.134612 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134552 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 11:23:50.134612 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.134563 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 11:23:50.135086 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.135064 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 11:23:50.135192 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.135114 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.234905 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.234867 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.237151 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.237131 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.237249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.237166 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.237249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.237177 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.237249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.237205 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.246773 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.246733 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.246773 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.246756 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-12.ec2.internal\": node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.252420 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.252397 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 11:23:50.253790 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.253765 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 11:23:50.253896 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.253811 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 11:23:50.253896 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.253832 2577 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 11:23:50.253896 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.253870 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 11:23:50.256616 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.256598 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:23:50.261197 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.261179 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.354028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.353983 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal"] Jul 22 11:23:50.354185 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.354090 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.355774 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.355756 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.355871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.355788 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.355871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.355804 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.357148 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357136 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.357273 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357259 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.357312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357292 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.357944 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357926 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.358042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357957 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.358042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357965 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.358042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357974 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.358042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357982 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.358042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.357995 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.359208 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.359194 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.359248 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.359220 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 11:23:50.359910 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.359887 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientMemory" Jul 22 11:23:50.360005 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.359934 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 11:23:50.360005 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.359946 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeHasSufficientPID" Jul 22 11:23:50.361285 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.361271 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.376528 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.376510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.376618 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.376532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.376618 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.376552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02a09a085ac5b89656342fe2b159dd5c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"02a09a085ac5b89656342fe2b159dd5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.390068 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.390046 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-12.ec2.internal\" not found" node="ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.394304 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.394289 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-12.ec2.internal\" not found" node="ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.462253 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.462225 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.477606 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477579 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.477606 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477601 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.477775 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.477775 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477646 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02a09a085ac5b89656342fe2b159dd5c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"02a09a085ac5b89656342fe2b159dd5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.477775 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477688 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/02a09a085ac5b89656342fe2b159dd5c-config\") pod \"kube-apiserver-proxy-ip-10-0-135-12.ec2.internal\" (UID: \"02a09a085ac5b89656342fe2b159dd5c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.477775 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.477717 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/73648c8ed4b6ddeeb1f8c0e9d7b3cd7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal\" (UID: \"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.563364 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.563285 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.663794 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.663765 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.693319 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.693290 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.698091 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:50.698074 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:50.763878 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.763840 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:50.864396 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:50.864320 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:52.772016 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:52.771984 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:52.821493 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:52.821463 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 11:23:52.821895 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:52.821647 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 11:23:52.872948 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:52.872911 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:52.899156 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:52.899124 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 11:18:51 +0000 UTC" deadline="2028-02-20 06:26:23.807752698 +0000 UTC" Jul 22 11:23:52.899156 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:52.899151 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13867h2m30.90860481s" Jul 22 11:23:52.973538 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:52.973471 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:53.006991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.006972 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:23:53.013444 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:53.013417 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a09a085ac5b89656342fe2b159dd5c.slice/crio-f3d7af982c0d113e662494a13bba67df09002965e0137c5cf900e4c4eb743e9d WatchSource:0}: Error finding container f3d7af982c0d113e662494a13bba67df09002965e0137c5cf900e4c4eb743e9d: Status 404 returned error can't find the container with id f3d7af982c0d113e662494a13bba67df09002965e0137c5cf900e4c4eb743e9d Jul 22 11:23:53.014222 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:53.014202 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73648c8ed4b6ddeeb1f8c0e9d7b3cd7a.slice/crio-32f43ace959e7ba3d9ee125d38be1b1de77771bb08afa3aa75899844b6c6964c WatchSource:0}: Error finding container 32f43ace959e7ba3d9ee125d38be1b1de77771bb08afa3aa75899844b6c6964c: Status 404 returned error can't find the container with id 32f43ace959e7ba3d9ee125d38be1b1de77771bb08afa3aa75899844b6c6964c Jul 22 11:23:53.020712 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.020694 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:23:53.063931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.063885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" event={"ID":"02a09a085ac5b89656342fe2b159dd5c","Type":"ContainerStarted","Data":"f3d7af982c0d113e662494a13bba67df09002965e0137c5cf900e4c4eb743e9d"} Jul 22 11:23:53.064843 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.064822 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a","Type":"ContainerStarted","Data":"32f43ace959e7ba3d9ee125d38be1b1de77771bb08afa3aa75899844b6c6964c"} Jul 22 11:23:53.073960 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.073939 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-12.ec2.internal\" not found" Jul 22 11:23:53.094460 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.094433 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:23:53.181931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.181912 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" Jul 22 11:23:53.196716 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.196688 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 11:23:53.198495 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.198481 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" Jul 22 11:23:53.210559 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.210540 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 11:23:53.413233 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.413154 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:23:53.662195 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.662158 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 11:23:53.862114 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.862046 2577 apiserver.go:52] "Watching apiserver" Jul 22 11:23:53.866917 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.866893 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 11:23:53.868898 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.868872 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-dhtvd","kube-system/global-pull-secret-syncer-5nfx5","openshift-cluster-node-tuning-operator/tuned-qb9w5","openshift-dns/node-resolver-dv9cn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal","openshift-multus/multus-additional-cni-plugins-mxfbc","openshift-multus/network-metrics-daemon-qhp5m","kube-system/konnectivity-agent-vkp9f","kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw","openshift-image-registry/node-ca-w4zp6","openshift-multus/multus-7sncc","openshift-network-diagnostics/network-check-target-dclsj","openshift-network-operator/iptables-alerter-f4c8v"] Jul 22 11:23:53.870501 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.870480 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:53.870616 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.870569 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:23:53.872210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.872186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.873407 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.873385 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.874487 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.874216 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.874487 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.874224 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 11:23:53.874487 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.874247 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.874487 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.874252 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5kh7h\"" Jul 22 11:23:53.874750 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.874530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.875082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.875067 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.875548 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.875528 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.875669 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.875645 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7qvls\"" Jul 22 11:23:53.875887 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.875866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.876563 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.876543 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rc97p\"" Jul 22 11:23:53.876690 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.876670 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.876690 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.876695 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.877012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.876993 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.877719 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.877701 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.877816 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.877757 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zcn57\"" Jul 22 11:23:53.877816 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.877804 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.878049 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.878026 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 11:23:53.878199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.878178 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.878890 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.878859 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.879229 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879210 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 11:23:53.879323 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879210 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 11:23:53.879323 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879282 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pjwrh\"" Jul 22 11:23:53.879424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879322 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:53.879424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879334 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.879517 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.879420 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:23:53.879564 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879517 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 11:23:53.880002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.879984 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bxlfx\"" Jul 22 11:23:53.880081 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.880015 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 11:23:53.880871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.880246 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 11:23:53.880871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.880634 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.880871 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.880692 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:23:53.881574 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.881556 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 11:23:53.882086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.882065 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.884692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.884190 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.884692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.884205 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.884692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.884294 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 11:23:53.884692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.884343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hbl5s\"" Jul 22 11:23:53.884692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.884481 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.887262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.886153 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 11:23:53.887262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.886176 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4cc9z\"" Jul 22 11:23:53.887262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.887155 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 11:23:53.888257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.887557 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 11:23:53.888257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.887977 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 11:23:53.889071 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.888757 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 11:23:53.889071 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.888761 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.890435 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.890414 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 11:23:53.890743 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.890727 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nz2gj\"" Jul 22 11:23:53.891612 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.891575 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 11:23:53.893978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.893945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-device-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.894068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.893984 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lrv7\" (UniqueName: \"kubernetes.io/projected/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-kube-api-access-6lrv7\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g559h\" (UniqueName: \"kubernetes.io/projected/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-kube-api-access-g559h\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.894068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.894068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894057 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-systemd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894081 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f1f798-5d67-4b30-b583-912ef1c3704e-tmp-dir\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894102 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-dbus\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894123 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-tmp\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894145 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894162 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p529v\" (UniqueName: \"kubernetes.io/projected/20f1f798-5d67-4b30-b583-912ef1c3704e-kube-api-access-p529v\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894176 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-run\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894210 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-konnectivity-ca\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.894240 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vjb2\" (UniqueName: \"kubernetes.io/projected/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kube-api-access-4vjb2\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894246 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovn-node-metrics-cert\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-os-release\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894335 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7p8f\" (UniqueName: \"kubernetes.io/projected/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-kube-api-access-z7p8f\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894356 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-sys\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-script-lib\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-host\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-agent-certs\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-kubelet\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894489 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894513 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-serviceca\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.894556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-kubelet-config\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894588 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysconfig\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894631 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-registration-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-env-overrides\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894673 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-sys-fs\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894688 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-var-lib-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894730 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-bin\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20f1f798-5d67-4b30-b583-912ef1c3704e-hosts-file\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894778 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-cnibin\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgggj\" (UniqueName: \"kubernetes.io/projected/b9e2624e-865c-4390-b5cc-8cd868bc24ba-kube-api-access-sgggj\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894828 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-host\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894852 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-slash\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894874 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-etc-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894892 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-ovn\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895084 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894905 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4e0775a-86eb-4d56-821b-b3aa96189071-host-slash\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-netns\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-log-socket\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.894987 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7g65\" (UniqueName: \"kubernetes.io/projected/3cb27479-56a8-4882-b005-805dfe1c934c-kube-api-access-v7g65\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895010 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-modprobe-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-conf\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895064 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895100 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/d4e0775a-86eb-4d56-821b-b3aa96189071-kube-api-access-sc7wj\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895140 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-system-cni-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895170 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895198 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-socket-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-node-log\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-netd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-config\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895323 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895337 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-kubernetes\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.895672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895350 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-systemd\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-systemd-units\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895424 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895437 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4e0775a-86eb-4d56-821b-b3aa96189071-iptables-alerter-script\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-lib-modules\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895476 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-var-lib-kubelet\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.896254 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.895492 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-tuned\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.901666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.901642 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 11:23:53.925378 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.925357 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cx49c" Jul 22 11:23:53.935204 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.935180 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cx49c" Jul 22 11:23:53.983545 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.983506 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 11:23:53.995739 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.995739 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995729 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/d4e0775a-86eb-4d56-821b-b3aa96189071-kube-api-access-sc7wj\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-system-cni-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-etc-kubernetes\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995837 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-system-cni-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.995922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-socket-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995943 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-node-log\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995966 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-netd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-config\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.995999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-kubernetes\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-systemd\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-k8s-cni-cncf-io\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-netd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-systemd-units\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996127 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-systemd\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996140 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-node-log\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996146 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-kubernetes\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996174 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-systemd-units\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.996206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996208 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4e0775a-86eb-4d56-821b-b3aa96189071-iptables-alerter-script\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996254 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-etc-selinux\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-lib-modules\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-var-lib-kubelet\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-tuned\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996331 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-device-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lrv7\" (UniqueName: \"kubernetes.io/projected/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-kube-api-access-6lrv7\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g559h\" (UniqueName: \"kubernetes.io/projected/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-kube-api-access-g559h\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996394 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-var-lib-kubelet\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996208 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996418 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-cnibin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-lib-modules\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996479 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-os-release\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996465 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996506 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-daemon-config\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.997033 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-systemd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f1f798-5d67-4b30-b583-912ef1c3704e-tmp-dir\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996635 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-dbus\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996670 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-config\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996732 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-socket-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996739 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-device-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996813 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996836 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996844 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4e0775a-86eb-4d56-821b-b3aa96189071-iptables-alerter-script\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996899 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-systemd\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20f1f798-5d67-4b30-b583-912ef1c3704e-tmp-dir\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-tmp\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997363 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p529v\" (UniqueName: \"kubernetes.io/projected/20f1f798-5d67-4b30-b583-912ef1c3704e-kube-api-access-p529v\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997488 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-run\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997527 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-konnectivity-ca\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997549 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-socket-dir-parent\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-dbus\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.997811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-run\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997639 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-multus\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.996994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cb27479-56a8-4882-b005-805dfe1c934c-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997695 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vjb2\" (UniqueName: \"kubernetes.io/projected/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kube-api-access-4vjb2\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997765 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovn-node-metrics-cert\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997789 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-os-release\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997812 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997837 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7p8f\" (UniqueName: \"kubernetes.io/projected/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-kube-api-access-z7p8f\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997891 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-sys\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997896 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-os-release\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-netns\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.997978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-script-lib\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.998031 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.998114 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:23:54.498071872 +0000 UTC m=+3.005972736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-host\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998160 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-agent-certs\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.998826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-hostroot\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998209 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-multus-certs\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.998215 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998254 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dqxh\" (UniqueName: \"kubernetes.io/projected/097d7dd7-5754-4f5f-a08f-8362cf166c10-kube-api-access-4dqxh\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-kubelet\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:53.998315 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:54.49827949 +0000 UTC m=+3.006180353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998321 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-kubelet\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998327 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-konnectivity-ca\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998321 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-sys\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998447 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998493 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-host\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998511 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-serviceca\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998551 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-kubelet-config\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998676 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysconfig\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998722 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-registration-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:53.999654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-env-overrides\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998770 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-sys-fs\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998822 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-var-lib-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998839 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998877 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-bin\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998879 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b184645a-a72b-4102-9927-a276ab76719d-kubelet-config\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998887 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysconfig\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998822 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovnkube-script-lib\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998916 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-sys-fs\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998949 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20f1f798-5d67-4b30-b583-912ef1c3704e-hosts-file\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998973 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74fbc3a6-7da0-4e15-8f0c-651d2689a341-registration-dir\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.998994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-cnibin\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999001 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-serviceca\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999024 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgggj\" (UniqueName: \"kubernetes.io/projected/b9e2624e-865c-4390-b5cc-8cd868bc24ba-kube-api-access-sgggj\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999029 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/20f1f798-5d67-4b30-b583-912ef1c3704e-hosts-file\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-host\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.000457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cb27479-56a8-4882-b005-805dfe1c934c-cnibin\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999097 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-cni-binary-copy\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-slash\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999132 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-host\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999166 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-slash\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999179 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-etc-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999204 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-ovn\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4e0775a-86eb-4d56-821b-b3aa96189071-host-slash\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999242 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-env-overrides\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999265 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4e0775a-86eb-4d56-821b-b3aa96189071-host-slash\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999270 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-cni-bin\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999265 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-system-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999291 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-var-lib-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999307 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999313 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-run-ovn\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-etc-openvswitch\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-netns\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999378 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-log-socket\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999412 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-host-run-netns\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7g65\" (UniqueName: \"kubernetes.io/projected/3cb27479-56a8-4882-b005-805dfe1c934c-kube-api-access-v7g65\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999462 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-modprobe-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999486 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-conf\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-bin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-log-socket\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-sysctl-conf\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-modprobe-d\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999685 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-kubelet\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:53.999737 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-conf-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.000674 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-etc-tuned\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.000819 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9e2624e-865c-4390-b5cc-8cd868bc24ba-tmp\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.001780 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.000854 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-ovn-node-metrics-cert\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.002438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.002419 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5d0a01d2-c2e2-4429-bdf4-420358a7c08b-agent-certs\") pod \"konnectivity-agent-vkp9f\" (UID: \"5d0a01d2-c2e2-4429-bdf4-420358a7c08b\") " pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:54.006718 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.006699 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:23:54.006853 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.006721 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:23:54.006853 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.006737 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:54.006853 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.006825 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:54.506805637 +0000 UTC m=+3.014706524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:54.009268 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.009123 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7p8f\" (UniqueName: \"kubernetes.io/projected/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-kube-api-access-z7p8f\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:54.010153 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.009644 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgggj\" (UniqueName: \"kubernetes.io/projected/b9e2624e-865c-4390-b5cc-8cd868bc24ba-kube-api-access-sgggj\") pod \"tuned-qb9w5\" (UID: \"b9e2624e-865c-4390-b5cc-8cd868bc24ba\") " pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.010269 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.010223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc7wj\" (UniqueName: \"kubernetes.io/projected/d4e0775a-86eb-4d56-821b-b3aa96189071-kube-api-access-sc7wj\") pod \"iptables-alerter-f4c8v\" (UID: \"d4e0775a-86eb-4d56-821b-b3aa96189071\") " pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:54.010355 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.010324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lrv7\" (UniqueName: \"kubernetes.io/projected/8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae-kube-api-access-6lrv7\") pod \"ovnkube-node-dhtvd\" (UID: \"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae\") " pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.011090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.011068 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vjb2\" (UniqueName: \"kubernetes.io/projected/74fbc3a6-7da0-4e15-8f0c-651d2689a341-kube-api-access-4vjb2\") pod \"aws-ebs-csi-driver-node-7bwtw\" (UID: \"74fbc3a6-7da0-4e15-8f0c-651d2689a341\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:54.011872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.011851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p529v\" (UniqueName: \"kubernetes.io/projected/20f1f798-5d67-4b30-b583-912ef1c3704e-kube-api-access-p529v\") pod \"node-resolver-dv9cn\" (UID: \"20f1f798-5d67-4b30-b583-912ef1c3704e\") " pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:54.012101 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.012080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7g65\" (UniqueName: \"kubernetes.io/projected/3cb27479-56a8-4882-b005-805dfe1c934c-kube-api-access-v7g65\") pod \"multus-additional-cni-plugins-mxfbc\" (UID: \"3cb27479-56a8-4882-b005-805dfe1c934c\") " pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:54.012932 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.012912 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g559h\" (UniqueName: \"kubernetes.io/projected/16921d4b-0b1f-4d6f-afdd-cc7aa184e095-kube-api-access-g559h\") pod \"node-ca-w4zp6\" (UID: \"16921d4b-0b1f-4d6f-afdd-cc7aa184e095\") " pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:54.100274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100236 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-cnibin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-os-release\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-daemon-config\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-socket-dir-parent\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100371 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-cnibin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-os-release\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100375 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-multus\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100419 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-multus\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-netns\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100469 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-socket-dir-parent\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-hostroot\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100506 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-hostroot\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100511 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-netns\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100514 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-multus-certs\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100545 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-multus-certs\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100555 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dqxh\" (UniqueName: \"kubernetes.io/projected/097d7dd7-5754-4f5f-a08f-8362cf166c10-kube-api-access-4dqxh\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100615 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-cni-binary-copy\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100642 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-system-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100666 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-bin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100737 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-cni-bin\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100740 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-system-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100766 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-kubelet\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.100806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100793 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-conf-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100819 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-etc-kubernetes\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-var-lib-kubelet\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-cni-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-k8s-cni-cncf-io\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-conf-dir\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100884 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-host-run-k8s-cni-cncf-io\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.100886 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/097d7dd7-5754-4f5f-a08f-8362cf166c10-etc-kubernetes\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.101004 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-multus-daemon-config\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.101345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.101153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/097d7dd7-5754-4f5f-a08f-8362cf166c10-cni-binary-copy\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.121527 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.121457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dqxh\" (UniqueName: \"kubernetes.io/projected/097d7dd7-5754-4f5f-a08f-8362cf166c10-kube-api-access-4dqxh\") pod \"multus-7sncc\" (UID: \"097d7dd7-5754-4f5f-a08f-8362cf166c10\") " pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.183868 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.183831 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w4zp6" Jul 22 11:23:54.194245 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.194220 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" Jul 22 11:23:54.208219 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.208192 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dv9cn" Jul 22 11:23:54.213761 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.213738 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f4c8v" Jul 22 11:23:54.219533 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.219511 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" Jul 22 11:23:54.226180 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.226159 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:23:54.232488 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.232469 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" Jul 22 11:23:54.239687 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.239671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:23:54.244916 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.244898 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7sncc" Jul 22 11:23:54.504253 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.504190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:54.504253 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.504221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:54.504409 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.504349 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:54.504409 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.504349 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:54.504470 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.504412 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:23:55.50439354 +0000 UTC m=+4.012294416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:54.504470 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.504426 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:55.504419899 +0000 UTC m=+4.012320763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:54.515264 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.515237 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fbc3a6_7da0_4e15_8f0c_651d2689a341.slice/crio-ea15025b5906af2f41a0ec9b8a2e4e4346f61f047765261b70a94433df1ed9b1 WatchSource:0}: Error finding container ea15025b5906af2f41a0ec9b8a2e4e4346f61f047765261b70a94433df1ed9b1: Status 404 returned error can't find the container with id ea15025b5906af2f41a0ec9b8a2e4e4346f61f047765261b70a94433df1ed9b1 Jul 22 11:23:54.516677 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.516537 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e2624e_865c_4390_b5cc_8cd868bc24ba.slice/crio-b068367f79bd386ca478d8f9fa85bb7a8012161f50629233f38a0e878435c898 WatchSource:0}: Error finding container b068367f79bd386ca478d8f9fa85bb7a8012161f50629233f38a0e878435c898: Status 404 returned error can't find the container with id b068367f79bd386ca478d8f9fa85bb7a8012161f50629233f38a0e878435c898 Jul 22 11:23:54.518159 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.518067 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e0775a_86eb_4d56_821b_b3aa96189071.slice/crio-2cc69d3c9a0bb0c66f0c7bd2e551a64e237f42c1d9613291c71b806da61e5279 WatchSource:0}: Error finding container 2cc69d3c9a0bb0c66f0c7bd2e551a64e237f42c1d9613291c71b806da61e5279: Status 404 returned error can't find the container with id 2cc69d3c9a0bb0c66f0c7bd2e551a64e237f42c1d9613291c71b806da61e5279 Jul 22 11:23:54.520979 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.520958 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16921d4b_0b1f_4d6f_afdd_cc7aa184e095.slice/crio-f4087156f1356ca4ae50b19fb0b72b41f20f424933a030e904165fde44276646 WatchSource:0}: Error finding container f4087156f1356ca4ae50b19fb0b72b41f20f424933a030e904165fde44276646: Status 404 returned error can't find the container with id f4087156f1356ca4ae50b19fb0b72b41f20f424933a030e904165fde44276646 Jul 22 11:23:54.522374 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.522349 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb27479_56a8_4882_b005_805dfe1c934c.slice/crio-da3f2ee564f78b93d612363fefb6c5ca455bbd0b0745911bf4342d12d3e5c8ef WatchSource:0}: Error finding container da3f2ee564f78b93d612363fefb6c5ca455bbd0b0745911bf4342d12d3e5c8ef: Status 404 returned error can't find the container with id da3f2ee564f78b93d612363fefb6c5ca455bbd0b0745911bf4342d12d3e5c8ef Jul 22 11:23:54.522580 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.522555 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f1f798_5d67_4b30_b583_912ef1c3704e.slice/crio-2e5401c6d0a21f061ed89350b96d5277ce2567c9af5fbd71146f8c07062f2df9 WatchSource:0}: Error finding container 2e5401c6d0a21f061ed89350b96d5277ce2567c9af5fbd71146f8c07062f2df9: Status 404 returned error can't find the container with id 2e5401c6d0a21f061ed89350b96d5277ce2567c9af5fbd71146f8c07062f2df9 Jul 22 11:23:54.523457 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.523342 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8bada0_b5c1_4f11_a27d_c86e5f1d82ae.slice/crio-dea28e440c6e7a185028d7ac00fe0f884c8f4734f6063ce04426ae9d2181365d WatchSource:0}: Error finding container dea28e440c6e7a185028d7ac00fe0f884c8f4734f6063ce04426ae9d2181365d: Status 404 returned error can't find the container with id dea28e440c6e7a185028d7ac00fe0f884c8f4734f6063ce04426ae9d2181365d Jul 22 11:23:54.524964 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.524909 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0a01d2_c2e2_4429_bdf4_420358a7c08b.slice/crio-4fc218da9d45cf3b79f881a020b097b9fa09b6c5d334e69e4814fae3fefb623b WatchSource:0}: Error finding container 4fc218da9d45cf3b79f881a020b097b9fa09b6c5d334e69e4814fae3fefb623b: Status 404 returned error can't find the container with id 4fc218da9d45cf3b79f881a020b097b9fa09b6c5d334e69e4814fae3fefb623b Jul 22 11:23:54.527181 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:23:54.527160 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097d7dd7_5754_4f5f_a08f_8362cf166c10.slice/crio-5865ab09fb56b16841985ce751b24ac7b9cae6a6ae4d5a5c01b10da0795a3e2f WatchSource:0}: Error finding container 5865ab09fb56b16841985ce751b24ac7b9cae6a6ae4d5a5c01b10da0795a3e2f: Status 404 returned error can't find the container with id 5865ab09fb56b16841985ce751b24ac7b9cae6a6ae4d5a5c01b10da0795a3e2f Jul 22 11:23:54.605192 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.605163 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:54.605336 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.605318 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:23:54.605376 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.605342 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:23:54.605376 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.605353 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:54.605446 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:54.605401 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:55.60538629 +0000 UTC m=+4.113287154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:54.937271 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.937071 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:18:53 +0000 UTC" deadline="2027-12-29 22:49:20.714354643 +0000 UTC" Jul 22 11:23:54.937271 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:54.937270 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12611h25m25.777089261s" Jul 22 11:23:55.062347 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.061630 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:55.062347 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.061777 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:23:55.062347 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.062184 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:55.062347 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.062296 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:23:55.098325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.098218 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7sncc" event={"ID":"097d7dd7-5754-4f5f-a08f-8362cf166c10","Type":"ContainerStarted","Data":"5865ab09fb56b16841985ce751b24ac7b9cae6a6ae4d5a5c01b10da0795a3e2f"} Jul 22 11:23:55.103725 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.103685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"dea28e440c6e7a185028d7ac00fe0f884c8f4734f6063ce04426ae9d2181365d"} Jul 22 11:23:55.106639 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.106610 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerStarted","Data":"da3f2ee564f78b93d612363fefb6c5ca455bbd0b0745911bf4342d12d3e5c8ef"} Jul 22 11:23:55.112630 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.112532 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w4zp6" event={"ID":"16921d4b-0b1f-4d6f-afdd-cc7aa184e095","Type":"ContainerStarted","Data":"f4087156f1356ca4ae50b19fb0b72b41f20f424933a030e904165fde44276646"} Jul 22 11:23:55.118420 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.118328 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" event={"ID":"b9e2624e-865c-4390-b5cc-8cd868bc24ba","Type":"ContainerStarted","Data":"b068367f79bd386ca478d8f9fa85bb7a8012161f50629233f38a0e878435c898"} Jul 22 11:23:55.120736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.120654 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" event={"ID":"74fbc3a6-7da0-4e15-8f0c-651d2689a341","Type":"ContainerStarted","Data":"ea15025b5906af2f41a0ec9b8a2e4e4346f61f047765261b70a94433df1ed9b1"} Jul 22 11:23:55.130029 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.129832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkp9f" event={"ID":"5d0a01d2-c2e2-4429-bdf4-420358a7c08b","Type":"ContainerStarted","Data":"4fc218da9d45cf3b79f881a020b097b9fa09b6c5d334e69e4814fae3fefb623b"} Jul 22 11:23:55.133584 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.133513 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dv9cn" event={"ID":"20f1f798-5d67-4b30-b583-912ef1c3704e","Type":"ContainerStarted","Data":"2e5401c6d0a21f061ed89350b96d5277ce2567c9af5fbd71146f8c07062f2df9"} Jul 22 11:23:55.141124 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.141096 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f4c8v" event={"ID":"d4e0775a-86eb-4d56-821b-b3aa96189071","Type":"ContainerStarted","Data":"2cc69d3c9a0bb0c66f0c7bd2e551a64e237f42c1d9613291c71b806da61e5279"} Jul 22 11:23:55.145381 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.144682 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" event={"ID":"02a09a085ac5b89656342fe2b159dd5c","Type":"ContainerStarted","Data":"242fb02d96c1eae6fc181fa5e2c9dfe5acc1c90763e3cc3884bc8cfadc89b3f7"} Jul 22 11:23:55.515733 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.515647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:55.515733 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.515698 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:55.515945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.515857 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:55.515945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.515934 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:57.515913425 +0000 UTC m=+6.023814304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:55.516368 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.516345 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:55.516427 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.516399 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:23:57.516384513 +0000 UTC m=+6.024285393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:55.617306 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.616628 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:55.617306 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.616840 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:23:55.617306 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.616862 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:23:55.617306 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.616877 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:55.617306 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:55.616936 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:23:57.616918344 +0000 UTC m=+6.124819234 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:55.938195 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.938083 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 11:18:53 +0000 UTC" deadline="2028-01-29 05:42:01.420996409 +0000 UTC" Jul 22 11:23:55.938195 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:55.938148 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13338h18m5.482851798s" Jul 22 11:23:56.062838 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:56.062804 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:56.063012 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:56.062944 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:23:57.062106 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:57.062074 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:57.062564 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:57.062087 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:57.062564 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.062210 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:23:57.062564 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.062283 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:23:57.536910 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:57.536826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:57.536910 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:57.536878 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:57.537141 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.537024 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:57.537141 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.537046 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:57.537141 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.537105 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:24:01.537082627 +0000 UTC m=+10.044983506 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:23:57.537141 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.537127 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:01.537116183 +0000 UTC m=+10.045017062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:23:57.638836 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:57.638798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:57.639020 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.638999 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:23:57.639079 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.639031 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:23:57.639079 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.639046 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:57.639177 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:57.639110 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:01.639090761 +0000 UTC m=+10.146991644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:23:58.063639 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:58.063585 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:23:58.064075 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:58.063738 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:23:59.061436 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:59.061407 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:23:59.061657 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:59.061518 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:23:59.061909 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:23:59.061892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:23:59.062087 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:23:59.061997 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:00.062812 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:00.062775 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:00.063161 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:00.062920 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:01.062267 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:01.062230 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:01.062474 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:01.062277 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:01.062474 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.062376 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:01.062875 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.062832 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:01.574696 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:01.574648 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:01.574696 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:01.574702 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:01.574941 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.574822 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:01.574941 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.574831 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:01.574941 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.574894 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:24:09.574872756 +0000 UTC m=+18.082773636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:01.574941 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.574916 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:09.574904967 +0000 UTC m=+18.082805849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:01.675588 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:01.675535 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:01.675784 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.675753 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:24:01.675784 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.675774 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:24:01.675855 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.675786 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:01.675855 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:01.675846 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:09.675826569 +0000 UTC m=+18.183727448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:02.062789 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:02.062651 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:02.062789 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:02.062768 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:03.062208 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:03.061687 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:03.062208 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:03.061815 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:03.062208 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:03.061839 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:03.062208 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:03.061898 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:04.061837 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:04.061796 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:04.062299 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:04.061934 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:05.061433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:05.061398 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:05.061620 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:05.061528 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:05.061620 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:05.061584 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:05.061757 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:05.061732 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:06.061864 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:06.061774 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:06.062926 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:06.061924 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:07.062073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:07.062039 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:07.062073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:07.062081 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:07.062537 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:07.062207 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:07.062537 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:07.062356 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:08.062247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:08.062204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:08.062690 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:08.062328 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:09.061713 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:09.061676 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:09.061713 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:09.061692 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:09.061945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.061799 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:09.061945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.061917 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:09.637801 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:09.637707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:09.637801 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:09.637746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:09.638286 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.637864 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:09.638286 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.637864 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:09.638286 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.637929 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:24:25.637910625 +0000 UTC m=+34.145811503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:09.638286 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.637946 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:25.637936768 +0000 UTC m=+34.145837632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:09.738718 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:09.738668 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:09.738909 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.738872 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:24:09.738909 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.738902 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:24:09.739021 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.738917 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:09.739021 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:09.738982 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:25.73896632 +0000 UTC m=+34.246867185 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:10.061632 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:10.061538 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:10.061802 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:10.061692 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:11.062005 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:11.061966 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:11.062423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:11.062014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:11.062423 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:11.062102 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:11.062423 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:11.062203 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:12.062250 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:12.062204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:12.062744 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:12.062323 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:13.061783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:13.061750 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:13.061973 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:13.061787 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:13.061973 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:13.061859 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:13.062059 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:13.062000 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:14.061179 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:14.061155 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:14.061494 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:14.061286 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:15.062445 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.062145 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:15.062946 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.062171 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:15.062946 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:15.062531 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:15.062946 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:15.062616 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:15.189849 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.189820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7sncc" event={"ID":"097d7dd7-5754-4f5f-a08f-8362cf166c10","Type":"ContainerStarted","Data":"54a49e22b161f863edfdd8ed1ae01aed08148b148f9aff5e6b16988515a33ea2"} Jul 22 11:24:15.192402 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192383 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:24:15.192724 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192704 2577 generic.go:356] "Generic (PLEG): container finished" podID="8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae" containerID="fde3d0f4586d9280109f21e8477b8f163e45d7b641b1d73eda0ec3e749875738" exitCode=1 Jul 22 11:24:15.192810 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192763 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"3fdb4aaf9ed39cd2601b6d7dbc5938ccef7f652434b64123d9fe5ef87c6782cf"} Jul 22 11:24:15.192854 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192817 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"8cb10455ececbe9b39a54e3daf9cea0ce5136a9f16b48551cc58a3b5c7524090"} Jul 22 11:24:15.192904 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192891 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"2d13c3ea835eabbd97290707fdc58a3256779db2060b7124d50a8ca9dfae0e7d"} Jul 22 11:24:15.192939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192930 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"bd345fcb66e9276630e2daba895e50bb04231ac16be605478511bd0e5050a853"} Jul 22 11:24:15.193028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.192975 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerDied","Data":"fde3d0f4586d9280109f21e8477b8f163e45d7b641b1d73eda0ec3e749875738"} Jul 22 11:24:15.193028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.193019 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"edf1a7fcca5fdc5c5caf23a43cf428f5382acecd22b49d0d917cb37dc9033b59"} Jul 22 11:24:15.194298 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.194278 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="083d5a1f3ab9cbec7be924f8452e153e7a7c7089e6ef160b7a49c0c0682ae52b" exitCode=0 Jul 22 11:24:15.194387 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.194307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"083d5a1f3ab9cbec7be924f8452e153e7a7c7089e6ef160b7a49c0c0682ae52b"} Jul 22 11:24:15.195871 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.195840 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w4zp6" event={"ID":"16921d4b-0b1f-4d6f-afdd-cc7aa184e095","Type":"ContainerStarted","Data":"b3e223a58aef21bf16024bb673660332c5756580f9bc0b1230cc1eec1c29888c"} Jul 22 11:24:15.197279 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.197261 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" event={"ID":"b9e2624e-865c-4390-b5cc-8cd868bc24ba","Type":"ContainerStarted","Data":"5f34994c6da47e30600d19868fb2c9a801b9035dd038e2cbb4e1f51d1606d1b1"} Jul 22 11:24:15.198504 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.198482 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" event={"ID":"74fbc3a6-7da0-4e15-8f0c-651d2689a341","Type":"ContainerStarted","Data":"32d5a6fa328bc7a644a1382f6340ba00ea7343d05d4b1b1f7fc44cc4de1d240b"} Jul 22 11:24:15.199858 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.199835 2577 generic.go:356] "Generic (PLEG): container finished" podID="73648c8ed4b6ddeeb1f8c0e9d7b3cd7a" containerID="86539c48dd8ab5b88c21e11b31d090a93f5ac7490aa20c058ce073cdfca1e937" exitCode=0 Jul 22 11:24:15.199952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.199917 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a","Type":"ContainerDied","Data":"86539c48dd8ab5b88c21e11b31d090a93f5ac7490aa20c058ce073cdfca1e937"} Jul 22 11:24:15.204687 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.204664 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vkp9f" event={"ID":"5d0a01d2-c2e2-4429-bdf4-420358a7c08b","Type":"ContainerStarted","Data":"46f2d44a8c6eba2a3acc0f58e84634935fa3d593ca5c25ad5fd5bd26ce420ef7"} Jul 22 11:24:15.206158 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.206134 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dv9cn" event={"ID":"20f1f798-5d67-4b30-b583-912ef1c3704e","Type":"ContainerStarted","Data":"c793e2b8a28c1fb606e597e8215a9f39ce78b9405a402668f1a8468839dda164"} Jul 22 11:24:15.209057 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.208986 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7sncc" podStartSLOduration=3.675850874 podStartE2EDuration="23.208975681s" totalImagesPullingTime="19.533124807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:15.208467246 +0000 UTC m=+23.716368132" watchObservedRunningTime="2026-07-22 11:24:15.208975681 +0000 UTC m=+23.716876563" Jul 22 11:24:15.209634 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.209585 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-12.ec2.internal" podStartSLOduration=22.20957471 podStartE2EDuration="22.20957471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:23:55.165769999 +0000 UTC m=+3.673670886" watchObservedRunningTime="2026-07-22 11:24:15.20957471 +0000 UTC m=+23.717475632" Jul 22 11:24:15.224919 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.224875 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vkp9f" podStartSLOduration=8.392119804 podStartE2EDuration="23.224857047s" totalImagesPullingTime="14.832737243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:15.2240872 +0000 UTC m=+23.731988085" watchObservedRunningTime="2026-07-22 11:24:15.224857047 +0000 UTC m=+23.732757934" Jul 22 11:24:15.254762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.254709 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w4zp6" podStartSLOduration=3.72751067 podStartE2EDuration="23.254689114s" totalImagesPullingTime="19.527178444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:15.2536965 +0000 UTC m=+23.761597386" watchObservedRunningTime="2026-07-22 11:24:15.254689114 +0000 UTC m=+23.762590001" Jul 22 11:24:15.270332 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.270297 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dv9cn" podStartSLOduration=3.744286703 podStartE2EDuration="23.270282432s" totalImagesPullingTime="19.525995729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:15.267778161 +0000 UTC m=+23.775679046" watchObservedRunningTime="2026-07-22 11:24:15.270282432 +0000 UTC m=+23.778183318" Jul 22 11:24:15.326111 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:15.326072 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qb9w5" podStartSLOduration=3.793019701 podStartE2EDuration="23.326057516s" totalImagesPullingTime="19.533037815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:15.325741276 +0000 UTC m=+23.833642161" watchObservedRunningTime="2026-07-22 11:24:15.326057516 +0000 UTC m=+23.833958446" Jul 22 11:24:16.062167 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.062137 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:16.062375 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:16.062270 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:16.211997 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.211958 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" event={"ID":"73648c8ed4b6ddeeb1f8c0e9d7b3cd7a","Type":"ContainerStarted","Data":"56423700d583c0d115c02d6b96c7598aa927ea0579be240d1e65ab0cb3760fea"} Jul 22 11:24:16.213944 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.213919 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f4c8v" event={"ID":"d4e0775a-86eb-4d56-821b-b3aa96189071","Type":"ContainerStarted","Data":"0bc3bf49fd214c17e5b9ab1794672621a8b7013733ee84f3286521483776d4d6"} Jul 22 11:24:16.244343 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.244268 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-12.ec2.internal" podStartSLOduration=23.244250973 podStartE2EDuration="23.244250973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:16.228391439 +0000 UTC m=+24.736292328" watchObservedRunningTime="2026-07-22 11:24:16.244250973 +0000 UTC m=+24.752151862" Jul 22 11:24:16.260028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.260006 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 11:24:16.958830 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.958731 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T11:24:16.26002714Z","UUID":"c1030d5e-cd2e-4c2a-b769-de9a5d27f894","Handler":null,"Name":"","Endpoint":""} Jul 22 11:24:16.960561 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.960539 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 11:24:16.960708 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:16.960584 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 11:24:17.062272 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:17.062236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:17.062458 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:17.062361 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:17.062788 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:17.062760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:17.062888 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:17.062863 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:17.218807 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:17.218719 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" event={"ID":"74fbc3a6-7da0-4e15-8f0c-651d2689a341","Type":"ContainerStarted","Data":"a14280d0b1746c08aadd872b08cbd506d09379a7ea807819ca2e57a474d9b022"} Jul 22 11:24:18.061486 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.061448 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:18.061689 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:18.061585 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:18.223376 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.223325 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" event={"ID":"74fbc3a6-7da0-4e15-8f0c-651d2689a341","Type":"ContainerStarted","Data":"e5c12c2d2190c98396cbe35a906534887ab65ccf4740c555041dfc61e4cf74fd"} Jul 22 11:24:18.226817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.226793 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:24:18.227233 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.227208 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"24b1a2fa60f4b200f9f90462b35a85796ae6168610e28487970fa4b9549d2fbc"} Jul 22 11:24:18.241977 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.241927 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f4c8v" podStartSLOduration=11.40004864 podStartE2EDuration="26.241910085s" totalImagesPullingTime="14.841861445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:16.243997781 +0000 UTC m=+24.751898677" watchObservedRunningTime="2026-07-22 11:24:18.241910085 +0000 UTC m=+26.749810972" Jul 22 11:24:18.242398 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:18.242375 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7bwtw" podStartSLOduration=4.086860854 podStartE2EDuration="26.242367546s" totalImagesPullingTime="22.155506692s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:18.241709665 +0000 UTC m=+26.749610552" watchObservedRunningTime="2026-07-22 11:24:18.242367546 +0000 UTC m=+26.750268449" Jul 22 11:24:19.061237 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:19.061204 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:19.061415 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:19.061203 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:19.061415 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:19.061344 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:19.061519 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:19.061413 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:19.709588 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:19.709410 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:24:19.710271 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:19.710254 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:24:20.062321 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.062244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:20.062455 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:20.062346 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:20.234562 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.234537 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:24:20.234950 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.234928 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"b2064f22045b26f1ce9a7725d60ddc21764afa4aedfabe786fd72b198b6541db"} Jul 22 11:24:20.235198 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.235170 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:20.235347 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.235318 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:20.235415 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.235336 2577 scope.go:117] "RemoveContainer" containerID="fde3d0f4586d9280109f21e8477b8f163e45d7b641b1d73eda0ec3e749875738" Jul 22 11:24:20.237009 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.236973 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="4fe559415ddf24a2776dd102f40ae0e25441dd411616c14c97f384c68edd16b1" exitCode=0 Jul 22 11:24:20.237101 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.237040 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"4fe559415ddf24a2776dd102f40ae0e25441dd411616c14c97f384c68edd16b1"} Jul 22 11:24:20.237347 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.237329 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:24:20.238248 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.238232 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vkp9f" Jul 22 11:24:20.252269 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:20.252121 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:21.062098 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.062070 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:21.062623 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.062069 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:21.062623 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:21.062206 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:21.062623 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:21.062226 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:21.247447 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.247418 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:24:21.247922 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.247887 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" event={"ID":"8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae","Type":"ContainerStarted","Data":"b7d385bfc5c633faa6f41c0dfb02fc82c8cf1badac6ef028ddbd2fee2ebb99b3"} Jul 22 11:24:21.248335 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.248261 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:21.264644 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.264619 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:21.279257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.279206 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" podStartSLOduration=9.421135254 podStartE2EDuration="29.279193099s" totalImagesPullingTime="19.858057845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:21.278073217 +0000 UTC m=+29.785974099" watchObservedRunningTime="2026-07-22 11:24:21.279193099 +0000 UTC m=+29.787093984" Jul 22 11:24:21.360831 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.360649 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhp5m"] Jul 22 11:24:21.360831 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.360784 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:21.361102 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:21.360873 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:21.363630 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.363582 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nfx5"] Jul 22 11:24:21.363747 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.363710 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:21.363841 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:21.363818 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:21.364239 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.364218 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dclsj"] Jul 22 11:24:21.364346 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:21.364286 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:21.364400 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:21.364340 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:23.061294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:23.061251 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:23.061294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:23.061287 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:23.061866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:23.061373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:23.061866 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:23.061377 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:23.061866 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:23.061447 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:23.061866 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:23.061522 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:24.257217 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:24.257041 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="5941695664b1d70af5d2bcd9ce3ecaa268707f1872de9ed8c36a496d70def579" exitCode=0 Jul 22 11:24:24.257217 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:24.257117 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"5941695664b1d70af5d2bcd9ce3ecaa268707f1872de9ed8c36a496d70def579"} Jul 22 11:24:25.062047 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.062015 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:25.062047 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.062047 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:25.062299 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.062112 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dclsj" podUID="6e4ee25f-44d9-430b-ba57-356629e25597" Jul 22 11:24:25.062299 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.062144 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:25.062299 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.062252 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5nfx5" podUID="b184645a-a72b-4102-9927-a276ab76719d" Jul 22 11:24:25.062435 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.062349 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qhp5m" podUID="6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9" Jul 22 11:24:25.658470 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.658438 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:25.658470 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.658472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:25.659024 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.658577 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:25.659024 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.658622 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:25.659024 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.658653 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs podName:6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:57.658637211 +0000 UTC m=+66.166538075 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs") pod "network-metrics-daemon-qhp5m" (UID: "6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 11:24:25.659024 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.658687 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret podName:b184645a-a72b-4102-9927-a276ab76719d nodeName:}" failed. No retries permitted until 2026-07-22 11:24:57.658670115 +0000 UTC m=+66.166571006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret") pod "global-pull-secret-syncer-5nfx5" (UID: "b184645a-a72b-4102-9927-a276ab76719d") : object "kube-system"/"original-pull-secret" not registered Jul 22 11:24:25.759392 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:25.759312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:25.759543 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.759487 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 11:24:25.759543 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.759513 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 11:24:25.759543 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.759526 2577 projected.go:196] Error preparing data for projected volume kube-api-access-r6xfz for pod openshift-network-diagnostics/network-check-target-dclsj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:25.759727 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:25.759616 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz podName:6e4ee25f-44d9-430b-ba57-356629e25597 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:57.759570036 +0000 UTC m=+66.267470913 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r6xfz" (UniqueName: "kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz") pod "network-check-target-dclsj" (UID: "6e4ee25f-44d9-430b-ba57-356629e25597") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 11:24:26.264411 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.264374 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="34b6b43b777c66c20043d6afad61f237776eb771e6d58ce81ef95f325f05b5b7" exitCode=0 Jul 22 11:24:26.264567 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.264421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"34b6b43b777c66c20043d6afad61f237776eb771e6d58ce81ef95f325f05b5b7"} Jul 22 11:24:26.852909 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.852878 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-12.ec2.internal" event="NodeReady" Jul 22 11:24:26.853397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.853033 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 11:24:26.902866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.902736 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pbzxt"] Jul 22 11:24:26.905776 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.905749 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nhdf7"] Jul 22 11:24:26.905920 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.905883 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:26.907547 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.907528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:26.908476 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.908450 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 11:24:26.908476 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.908464 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 11:24:26.908768 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.908587 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqh28\"" Jul 22 11:24:26.909481 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.909440 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dccn9\"" Jul 22 11:24:26.909707 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.909688 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 11:24:26.910161 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.909922 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 11:24:26.910161 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.910002 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 11:24:26.919400 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.919340 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbzxt"] Jul 22 11:24:26.920217 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:26.920194 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nhdf7"] Jul 22 11:24:27.061972 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.061893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:27.062155 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.061908 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:27.062155 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.062037 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:27.064939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064738 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cs9j5\"" Jul 22 11:24:27.064939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064738 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 11:24:27.064939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064812 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 11:24:27.064939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064839 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7fc2b\"" Jul 22 11:24:27.064939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064854 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 11:24:27.065261 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.064991 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 11:24:27.068073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068052 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16572c34-03d5-47fa-9182-fbfc27acf6d5-config-volume\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.068191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068079 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16572c34-03d5-47fa-9182-fbfc27acf6d5-tmp-dir\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.068191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.068191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068131 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d624970-f8dc-4111-ba21-c57dca9f579f-cert\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.068295 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068234 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfflt\" (UniqueName: \"kubernetes.io/projected/8d624970-f8dc-4111-ba21-c57dca9f579f-kube-api-access-gfflt\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.068295 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.068272 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p782q\" (UniqueName: \"kubernetes.io/projected/16572c34-03d5-47fa-9182-fbfc27acf6d5-kube-api-access-p782q\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.169370 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.169556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169383 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d624970-f8dc-4111-ba21-c57dca9f579f-cert\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.169556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169423 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfflt\" (UniqueName: \"kubernetes.io/projected/8d624970-f8dc-4111-ba21-c57dca9f579f-kube-api-access-gfflt\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.169556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169452 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p782q\" (UniqueName: \"kubernetes.io/projected/16572c34-03d5-47fa-9182-fbfc27acf6d5-kube-api-access-p782q\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.169556 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.169547 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:27.169805 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.169647 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:27.669625505 +0000 UTC m=+36.177526384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:27.169805 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169687 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16572c34-03d5-47fa-9182-fbfc27acf6d5-config-volume\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.169805 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.169717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16572c34-03d5-47fa-9182-fbfc27acf6d5-tmp-dir\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.170120 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.170089 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16572c34-03d5-47fa-9182-fbfc27acf6d5-tmp-dir\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.170236 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.170172 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/16572c34-03d5-47fa-9182-fbfc27acf6d5-config-volume\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.174483 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.174457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d624970-f8dc-4111-ba21-c57dca9f579f-cert\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.180130 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.180104 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p782q\" (UniqueName: \"kubernetes.io/projected/16572c34-03d5-47fa-9182-fbfc27acf6d5-kube-api-access-p782q\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.180350 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.180332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfflt\" (UniqueName: \"kubernetes.io/projected/8d624970-f8dc-4111-ba21-c57dca9f579f-kube-api-access-gfflt\") pod \"ingress-canary-nhdf7\" (UID: \"8d624970-f8dc-4111-ba21-c57dca9f579f\") " pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.225351 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.225318 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nhdf7" Jul 22 11:24:27.379215 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.379019 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nhdf7"] Jul 22 11:24:27.384270 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:27.384238 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d624970_f8dc_4111_ba21_c57dca9f579f.slice/crio-d204f72fa66fca9efb6d2e9cdbe7bff64bf87f0dc4a73d1d830393b22118b45f WatchSource:0}: Error finding container d204f72fa66fca9efb6d2e9cdbe7bff64bf87f0dc4a73d1d830393b22118b45f: Status 404 returned error can't find the container with id d204f72fa66fca9efb6d2e9cdbe7bff64bf87f0dc4a73d1d830393b22118b45f Jul 22 11:24:27.565276 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.565235 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk"] Jul 22 11:24:27.568118 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.567956 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5568f487dd-hjg9f"] Jul 22 11:24:27.568118 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.568064 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.569896 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.569856 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.571653 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.572000 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.572414 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.573033 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pcgvv\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.573309 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.574370 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.574383 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 11:24:27.574549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.574444 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 11:24:27.575513 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.575441 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xpc2d\"" Jul 22 11:24:27.575626 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.575508 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 11:24:27.575626 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.575513 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 11:24:27.575626 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.575619 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 11:24:27.581567 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.581519 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk"] Jul 22 11:24:27.583507 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.583478 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5568f487dd-hjg9f"] Jul 22 11:24:27.675767 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.675732 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:27.675953 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.675782 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2gpt\" (UniqueName: \"kubernetes.io/projected/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-kube-api-access-b2gpt\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.675953 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.675905 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:27.676068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.675914 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-default-certificate\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.676068 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.675973 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:28.675953234 +0000 UTC m=+37.183854102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:27.676068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676006 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9jk7\" (UniqueName: \"kubernetes.io/projected/17631881-9aa5-4633-b964-ece76c82b2e1-kube-api-access-s9jk7\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.676068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.676298 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.676298 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676176 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.676298 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676251 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-stats-auth\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.676298 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.676278 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.776884 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.776843 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-stats-auth\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.777051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.776894 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.777051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.776994 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2gpt\" (UniqueName: \"kubernetes.io/projected/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-kube-api-access-b2gpt\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.777051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.777015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-default-certificate\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.777051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.777043 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9jk7\" (UniqueName: \"kubernetes.io/projected/17631881-9aa5-4633-b964-ece76c82b2e1-kube-api-access-s9jk7\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.777262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.777071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.777262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.777121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.777262 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.777144 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.777262 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.777236 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:27.777433 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.777272 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:28.277247441 +0000 UTC m=+36.785148309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:27.777433 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.777337 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:28.277318559 +0000 UTC m=+36.785219451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:27.777433 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.777341 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:27.777433 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:27.777392 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:28.277379213 +0000 UTC m=+36.785280081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:27.778290 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.778267 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:27.780494 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.780470 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-default-certificate\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.780578 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.780479 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-stats-auth\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.786237 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.786218 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9jk7\" (UniqueName: \"kubernetes.io/projected/17631881-9aa5-4633-b964-ece76c82b2e1-kube-api-access-s9jk7\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:27.786317 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:27.786266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2gpt\" (UniqueName: \"kubernetes.io/projected/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-kube-api-access-b2gpt\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:28.271356 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:28.271309 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nhdf7" event={"ID":"8d624970-f8dc-4111-ba21-c57dca9f579f","Type":"ContainerStarted","Data":"d204f72fa66fca9efb6d2e9cdbe7bff64bf87f0dc4a73d1d830393b22118b45f"} Jul 22 11:24:28.281865 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:28.281804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:28.282016 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:28.281889 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:28.282016 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.281955 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:28.282016 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:28.282004 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:28.282173 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.282023 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:29.2820036 +0000 UTC m=+37.789904482 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:28.282219 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.282195 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:29.282174826 +0000 UTC m=+37.790075703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:28.282302 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.282277 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:28.282369 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.282328 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:29.282309096 +0000 UTC m=+37.790209964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:28.686452 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:28.686420 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:28.686648 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.686578 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:28.686713 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:28.686664 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:30.686644405 +0000 UTC m=+39.194545291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:29.291304 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:29.291272 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:29.291304 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:29.291305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:29.291426 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:29.291426 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:29.291477 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:31.291462555 +0000 UTC m=+39.799363419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:29.291497 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:31.291485988 +0000 UTC m=+39.799386852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:29.291560 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:29.291793 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:29.291698 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:31.29168919 +0000 UTC m=+39.799590054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:30.702844 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:30.702807 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:30.703320 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:30.702957 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:30.703320 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:30.703019 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:34.703003567 +0000 UTC m=+43.210904431 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:31.307758 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:31.307717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:31.307769 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:31.307796 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:31.307867 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:35.307850343 +0000 UTC m=+43.815751206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:31.307897 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:31.307912 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:31.307930 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:31.307936 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:35.307925159 +0000 UTC m=+43.815826024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:31.308152 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:31.307963 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:35.307950201 +0000 UTC m=+43.815851078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:32.282811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.282659 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="2b3949a0879d72c00dd7b117d2ec686e2dffee0ca35069853b537bb542c690c1" exitCode=0 Jul 22 11:24:32.283273 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.282740 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"2b3949a0879d72c00dd7b117d2ec686e2dffee0ca35069853b537bb542c690c1"} Jul 22 11:24:32.284392 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.284368 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nhdf7" event={"ID":"8d624970-f8dc-4111-ba21-c57dca9f579f","Type":"ContainerStarted","Data":"02454fd0a330462d5d05c01071bd87a057b9074a6b6c896557930692f8326b18"} Jul 22 11:24:32.326045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.325981 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nhdf7" podStartSLOduration=2.247033998 podStartE2EDuration="6.32596017s" totalImagesPullingTime="4.078926172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:32.325034677 +0000 UTC m=+40.832935564" watchObservedRunningTime="2026-07-22 11:24:32.32596017 +0000 UTC m=+40.833861057" Jul 22 11:24:32.888027 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.887989 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5z47w"] Jul 22 11:24:32.893949 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.893922 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:32.897029 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.897004 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 11:24:32.897251 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.897234 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 11:24:32.897483 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.897468 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5pxx7\"" Jul 22 11:24:32.899521 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.899504 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 11:24:32.899822 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.899788 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 11:24:32.910519 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:32.910499 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5z47w"] Jul 22 11:24:33.021543 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.021502 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04ee0ab4-1873-4586-94a1-eb901c612e8c-data-volume\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.021543 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.021540 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.021783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.021660 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04ee0ab4-1873-4586-94a1-eb901c612e8c-crio-socket\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.021783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.021729 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.021783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.021756 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt4rj\" (UniqueName: \"kubernetes.io/projected/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-api-access-dt4rj\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122378 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122345 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04ee0ab4-1873-4586-94a1-eb901c612e8c-crio-socket\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt4rj\" (UniqueName: \"kubernetes.io/projected/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-api-access-dt4rj\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04ee0ab4-1873-4586-94a1-eb901c612e8c-data-volume\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122475 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122735 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.122570 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:24:33.122735 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122622 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/04ee0ab4-1873-4586-94a1-eb901c612e8c-crio-socket\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.122735 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.122630 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:33.122735 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.122652 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:33.622633412 +0000 UTC m=+42.130534278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : configmap "kube-rbac-proxy" not found Jul 22 11:24:33.122735 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.122716 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:33.62270037 +0000 UTC m=+42.130601236 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:33.122960 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.122834 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/04ee0ab4-1873-4586-94a1-eb901c612e8c-data-volume\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.134326 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.134302 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt4rj\" (UniqueName: \"kubernetes.io/projected/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-api-access-dt4rj\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.289530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.289444 2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb27479-56a8-4882-b005-805dfe1c934c" containerID="cd7a2638534a7e5ce4b171684d96fce07b0e6e660e56d7a058100eeba049edcd" exitCode=0 Jul 22 11:24:33.289999 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.289530 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerDied","Data":"cd7a2638534a7e5ce4b171684d96fce07b0e6e660e56d7a058100eeba049edcd"} Jul 22 11:24:33.627740 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.627706 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.627894 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:33.627779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:33.627894 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.627844 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:33.627961 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.627897 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:24:33.627961 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.627910 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:34.627894869 +0000 UTC m=+43.135795733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:33.627961 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:33.627953 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:34.627935795 +0000 UTC m=+43.135836696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : configmap "kube-rbac-proxy" not found Jul 22 11:24:34.294888 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.294848 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" event={"ID":"3cb27479-56a8-4882-b005-805dfe1c934c","Type":"ContainerStarted","Data":"f710995c9355564bfb85ad292ae68c84379682df0434609429e9c1359f86e881"} Jul 22 11:24:34.323772 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.323723 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mxfbc" podStartSLOduration=9.150046065 podStartE2EDuration="42.323708049s" totalImagesPullingTime="33.173661984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:34.322425799 +0000 UTC m=+42.830326695" watchObservedRunningTime="2026-07-22 11:24:34.323708049 +0000 UTC m=+42.831608948" Jul 22 11:24:34.635168 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.635138 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:34.635303 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.635238 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:34.635303 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.635261 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:24:34.635380 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.635325 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:36.635309962 +0000 UTC m=+45.143210828 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : configmap "kube-rbac-proxy" not found Jul 22 11:24:34.635380 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.635345 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:34.635449 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.635383 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:36.635372999 +0000 UTC m=+45.143273862 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:34.736017 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.735987 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:34.736192 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.736141 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:34.736248 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:34.736200 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:42.736185701 +0000 UTC m=+51.244086565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:34.868761 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:34.868732 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dv9cn_20f1f798-5d67-4b30-b583-912ef1c3704e/dns-node-resolver/0.log" Jul 22 11:24:35.341113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:35.341079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:35.341113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:35.341115 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:35.341257 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:35.341276 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:35.341331 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:43.341311597 +0000 UTC m=+51.849212461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:35.341357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:35.341417 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:43.341405403 +0000 UTC m=+51.849306267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:35.341621 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:35.341433 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:43.341424826 +0000 UTC m=+51.849325730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:35.868376 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:35.868353 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w4zp6_16921d4b-0b1f-4d6f-afdd-cc7aa184e095/node-ca/0.log" Jul 22 11:24:36.468816 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:36.468792 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nhdf7_8d624970-f8dc-4111-ba21-c57dca9f579f/serve-healthcheck-canary/0.log" Jul 22 11:24:36.651000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:36.650964 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:36.651118 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:36.651018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:36.651118 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:36.651107 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:24:36.651181 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:36.651118 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:36.651181 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:36.651165 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:40.651149739 +0000 UTC m=+49.159050611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : configmap "kube-rbac-proxy" not found Jul 22 11:24:36.651181 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:36.651181 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:40.651173845 +0000 UTC m=+49.159074709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:40.682027 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:40.681991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:40.682560 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:40.682046 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:40.682560 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:40.682107 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 11:24:40.682560 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:40.682153 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:48.682140115 +0000 UTC m=+57.190040980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : configmap "kube-rbac-proxy" not found Jul 22 11:24:40.682560 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:40.682158 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:40.682560 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:40.682210 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:24:48.682197144 +0000 UTC m=+57.190098007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:42.799721 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:42.799677 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:42.800115 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:42.799834 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 11:24:42.800115 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:42.799899 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls podName:16572c34-03d5-47fa-9182-fbfc27acf6d5 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:58.799883914 +0000 UTC m=+67.307784778 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls") pod "dns-default-pbzxt" (UID: "16572c34-03d5-47fa-9182-fbfc27acf6d5") : secret "dns-default-metrics-tls" not found Jul 22 11:24:43.403516 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:43.403477 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:43.403736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:43.403543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:43.403736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:43.403568 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:43.403736 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:43.403657 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:59.403639711 +0000 UTC m=+67.911540575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : configmap references non-existent config key: service-ca.crt Jul 22 11:24:43.403736 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:43.403692 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:43.403736 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:43.403699 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 11:24:43.403945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:43.403745 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls podName:4fc79d4d-6476-4c68-8ed8-4716cd450a4b nodeName:}" failed. No retries permitted until 2026-07-22 11:24:59.403733363 +0000 UTC m=+67.911634226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-l92nk" (UID: "4fc79d4d-6476-4c68-8ed8-4716cd450a4b") : secret "cluster-monitoring-operator-tls" not found Jul 22 11:24:43.403945 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:43.403759 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs podName:17631881-9aa5-4633-b964-ece76c82b2e1 nodeName:}" failed. No retries permitted until 2026-07-22 11:24:59.403752714 +0000 UTC m=+67.911653578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs") pod "router-default-5568f487dd-hjg9f" (UID: "17631881-9aa5-4633-b964-ece76c82b2e1") : secret "router-metrics-certs-default" not found Jul 22 11:24:46.258472 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.258439 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-r2s55"] Jul 22 11:24:46.278735 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.278712 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-r2s55"] Jul 22 11:24:46.278866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.278798 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.281525 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.281502 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 11:24:46.281525 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.281517 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 11:24:46.282144 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.282125 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 11:24:46.282228 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.282169 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qhww6\"" Jul 22 11:24:46.282228 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.282128 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 11:24:46.326741 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.326716 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-key\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.326848 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.326831 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fsnv\" (UniqueName: \"kubernetes.io/projected/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-kube-api-access-7fsnv\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.326905 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.326891 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-cabundle\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.428101 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.428075 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-cabundle\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.428199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.428111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-key\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.428199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.428187 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fsnv\" (UniqueName: \"kubernetes.io/projected/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-kube-api-access-7fsnv\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.428713 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.428687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-cabundle\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.430610 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.430576 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-signing-key\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.438589 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.438570 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fsnv\" (UniqueName: \"kubernetes.io/projected/e2c28f2b-1138-41f4-840b-8b3932a7a9d7-kube-api-access-7fsnv\") pod \"service-ca-58598695f7-r2s55\" (UID: \"e2c28f2b-1138-41f4-840b-8b3932a7a9d7\") " pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.587359 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.587287 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-r2s55" Jul 22 11:24:46.731727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:46.731694 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-r2s55"] Jul 22 11:24:46.735282 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:46.735253 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c28f2b_1138_41f4_840b_8b3932a7a9d7.slice/crio-5067ab9ad4e0b91affb9b00ce90eec20e30661e49ff064bcd5da8905e69e5c4e WatchSource:0}: Error finding container 5067ab9ad4e0b91affb9b00ce90eec20e30661e49ff064bcd5da8905e69e5c4e: Status 404 returned error can't find the container with id 5067ab9ad4e0b91affb9b00ce90eec20e30661e49ff064bcd5da8905e69e5c4e Jul 22 11:24:47.325390 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:47.325355 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-r2s55" event={"ID":"e2c28f2b-1138-41f4-840b-8b3932a7a9d7","Type":"ContainerStarted","Data":"5067ab9ad4e0b91affb9b00ce90eec20e30661e49ff064bcd5da8905e69e5c4e"} Jul 22 11:24:48.747557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:48.747528 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:48.747931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:48.747629 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:48.747931 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:48.747689 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 11:24:48.747931 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:24:48.747749 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls podName:04ee0ab4-1873-4586-94a1-eb901c612e8c nodeName:}" failed. No retries permitted until 2026-07-22 11:25:04.747733931 +0000 UTC m=+73.255634794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5z47w" (UID: "04ee0ab4-1873-4586-94a1-eb901c612e8c") : secret "insights-runtime-extractor-tls" not found Jul 22 11:24:48.748187 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:48.748164 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/04ee0ab4-1873-4586-94a1-eb901c612e8c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:24:49.331373 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:49.331331 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-r2s55" event={"ID":"e2c28f2b-1138-41f4-840b-8b3932a7a9d7","Type":"ContainerStarted","Data":"b50006ed054e6e8eca4c2ac19bbd396501da090b980cd5581d2a80b76e3c2727"} Jul 22 11:24:49.348742 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:49.348690 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-r2s55" podStartSLOduration=1.387624833 podStartE2EDuration="3.348672001s" totalImagesPullingTime="1.961047168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:24:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:24:49.348050022 +0000 UTC m=+57.855950909" watchObservedRunningTime="2026-07-22 11:24:49.348672001 +0000 UTC m=+57.856572890" Jul 22 11:24:53.270743 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:53.270717 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dhtvd" Jul 22 11:24:57.723534 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.723493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:57.723534 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.723534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:57.726542 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.726519 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 11:24:57.726944 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.726926 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 11:24:57.736969 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.736937 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9-metrics-certs\") pod \"network-metrics-daemon-qhp5m\" (UID: \"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9\") " pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:57.736969 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.736965 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b184645a-a72b-4102-9927-a276ab76719d-original-pull-secret\") pod \"global-pull-secret-syncer-5nfx5\" (UID: \"b184645a-a72b-4102-9927-a276ab76719d\") " pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:57.824793 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.824760 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:57.827510 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.827484 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 11:24:57.837601 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.837576 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 11:24:57.847928 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.847891 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6xfz\" (UniqueName: \"kubernetes.io/projected/6e4ee25f-44d9-430b-ba57-356629e25597-kube-api-access-r6xfz\") pod \"network-check-target-dclsj\" (UID: \"6e4ee25f-44d9-430b-ba57-356629e25597\") " pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:57.976257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.976178 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cs9j5\"" Jul 22 11:24:57.980988 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.980966 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5nfx5" Jul 22 11:24:57.984676 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.984659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qhp5m" Jul 22 11:24:57.990376 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.990357 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7fc2b\"" Jul 22 11:24:57.998703 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:57.998443 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:24:58.160943 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.160089 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qhp5m"] Jul 22 11:24:58.164450 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:58.164419 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ab5f7d1_d0fb_4fbf_80b8_4674c6a48db9.slice/crio-16173cba0ded2224b3091775544ba31c5e1ea2b4446682399927c319b9dea395 WatchSource:0}: Error finding container 16173cba0ded2224b3091775544ba31c5e1ea2b4446682399927c319b9dea395: Status 404 returned error can't find the container with id 16173cba0ded2224b3091775544ba31c5e1ea2b4446682399927c319b9dea395 Jul 22 11:24:58.174407 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.174383 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5nfx5"] Jul 22 11:24:58.177657 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:58.177635 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb184645a_a72b_4102_9927_a276ab76719d.slice/crio-8ddce31e41631575e74e155ebb4f22ae27898133445a5046f2f6e436cb95b428 WatchSource:0}: Error finding container 8ddce31e41631575e74e155ebb4f22ae27898133445a5046f2f6e436cb95b428: Status 404 returned error can't find the container with id 8ddce31e41631575e74e155ebb4f22ae27898133445a5046f2f6e436cb95b428 Jul 22 11:24:58.196326 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.196295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dclsj"] Jul 22 11:24:58.202903 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:58.202883 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e4ee25f_44d9_430b_ba57_356629e25597.slice/crio-a2c282816578613202d8106f98c3830b2a0d5cb39ecae8383acc6da8c243803f WatchSource:0}: Error finding container a2c282816578613202d8106f98c3830b2a0d5cb39ecae8383acc6da8c243803f: Status 404 returned error can't find the container with id a2c282816578613202d8106f98c3830b2a0d5cb39ecae8383acc6da8c243803f Jul 22 11:24:58.353287 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.353250 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nfx5" event={"ID":"b184645a-a72b-4102-9927-a276ab76719d","Type":"ContainerStarted","Data":"8ddce31e41631575e74e155ebb4f22ae27898133445a5046f2f6e436cb95b428"} Jul 22 11:24:58.354282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.354257 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dclsj" event={"ID":"6e4ee25f-44d9-430b-ba57-356629e25597","Type":"ContainerStarted","Data":"a2c282816578613202d8106f98c3830b2a0d5cb39ecae8383acc6da8c243803f"} Jul 22 11:24:58.355308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.355290 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhp5m" event={"ID":"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9","Type":"ContainerStarted","Data":"16173cba0ded2224b3091775544ba31c5e1ea2b4446682399927c319b9dea395"} Jul 22 11:24:58.834385 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.834310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:58.837631 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:58.837567 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/16572c34-03d5-47fa-9182-fbfc27acf6d5-metrics-tls\") pod \"dns-default-pbzxt\" (UID: \"16572c34-03d5-47fa-9182-fbfc27acf6d5\") " pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:59.019964 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.019927 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bqh28\"" Jul 22 11:24:59.027909 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.027879 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pbzxt" Jul 22 11:24:59.324484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.324455 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pbzxt"] Jul 22 11:24:59.328045 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:59.328005 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16572c34_03d5_47fa_9182_fbfc27acf6d5.slice/crio-dd25638879afe50a2f2c04dc502d1fa72c37cdf4794f7ac06407304893d5933c WatchSource:0}: Error finding container dd25638879afe50a2f2c04dc502d1fa72c37cdf4794f7ac06407304893d5933c: Status 404 returned error can't find the container with id dd25638879afe50a2f2c04dc502d1fa72c37cdf4794f7ac06407304893d5933c Jul 22 11:24:59.361369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.361335 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhp5m" event={"ID":"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9","Type":"ContainerStarted","Data":"b6cb4c834a53155dbf1842316f25cab39a93c913a7cd7c3fe1ce261f87bec345"} Jul 22 11:24:59.363199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.363173 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzxt" event={"ID":"16572c34-03d5-47fa-9182-fbfc27acf6d5","Type":"ContainerStarted","Data":"dd25638879afe50a2f2c04dc502d1fa72c37cdf4794f7ac06407304893d5933c"} Jul 22 11:24:59.439693 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.439658 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:59.439844 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.439707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:59.439844 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.439779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:59.440531 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.440506 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17631881-9aa5-4633-b964-ece76c82b2e1-service-ca-bundle\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:59.443939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.443914 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17631881-9aa5-4633-b964-ece76c82b2e1-metrics-certs\") pod \"router-default-5568f487dd-hjg9f\" (UID: \"17631881-9aa5-4633-b964-ece76c82b2e1\") " pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:59.444045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.443913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4fc79d4d-6476-4c68-8ed8-4716cd450a4b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-l92nk\" (UID: \"4fc79d4d-6476-4c68-8ed8-4716cd450a4b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:59.686985 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.686902 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-pcgvv\"" Jul 22 11:24:59.693621 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.693402 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xpc2d\"" Jul 22 11:24:59.694437 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.694405 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" Jul 22 11:24:59.701490 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.701458 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:24:59.901289 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.901257 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk"] Jul 22 11:24:59.910737 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:59.910705 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc79d4d_6476_4c68_8ed8_4716cd450a4b.slice/crio-895d2cc2ccf61dce39f887f0b9c53d5430791dae1da64f37fc242aef35e6c0db WatchSource:0}: Error finding container 895d2cc2ccf61dce39f887f0b9c53d5430791dae1da64f37fc242aef35e6c0db: Status 404 returned error can't find the container with id 895d2cc2ccf61dce39f887f0b9c53d5430791dae1da64f37fc242aef35e6c0db Jul 22 11:24:59.927315 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:24:59.927111 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5568f487dd-hjg9f"] Jul 22 11:24:59.937992 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:24:59.934062 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17631881_9aa5_4633_b964_ece76c82b2e1.slice/crio-d4ef1cb8b95dc6f31956ddfa2651e6d8bbcde4a10e41ad0d75ca1e75c52507d3 WatchSource:0}: Error finding container d4ef1cb8b95dc6f31956ddfa2651e6d8bbcde4a10e41ad0d75ca1e75c52507d3: Status 404 returned error can't find the container with id d4ef1cb8b95dc6f31956ddfa2651e6d8bbcde4a10e41ad0d75ca1e75c52507d3 Jul 22 11:25:00.372128 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.372085 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qhp5m" event={"ID":"6ab5f7d1-d0fb-4fbf-80b8-4674c6a48db9","Type":"ContainerStarted","Data":"3d7b9d2bf10eca30f4287b5b500e2fc1c6cd464137d9cb99c7d362bd36777c85"} Jul 22 11:25:00.375835 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.375800 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5568f487dd-hjg9f" event={"ID":"17631881-9aa5-4633-b964-ece76c82b2e1","Type":"ContainerStarted","Data":"5a7ab0d4dad4c224e7793779e9b60e50acef9e576d3e5ac90d9a2aefbfc0a28a"} Jul 22 11:25:00.375990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.375858 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5568f487dd-hjg9f" event={"ID":"17631881-9aa5-4633-b964-ece76c82b2e1","Type":"ContainerStarted","Data":"d4ef1cb8b95dc6f31956ddfa2651e6d8bbcde4a10e41ad0d75ca1e75c52507d3"} Jul 22 11:25:00.378769 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.378716 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" event={"ID":"4fc79d4d-6476-4c68-8ed8-4716cd450a4b","Type":"ContainerStarted","Data":"895d2cc2ccf61dce39f887f0b9c53d5430791dae1da64f37fc242aef35e6c0db"} Jul 22 11:25:00.391674 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.391625 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qhp5m" podStartSLOduration=67.360553426 podStartE2EDuration="1m8.391581071s" totalImagesPullingTime="1.031027645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:00.390404271 +0000 UTC m=+68.898305160" watchObservedRunningTime="2026-07-22 11:25:00.391581071 +0000 UTC m=+68.899481962" Jul 22 11:25:00.411907 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.411708 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5568f487dd-hjg9f" podStartSLOduration=33.411688438 podStartE2EDuration="33.411688438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:24:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:00.410089822 +0000 UTC m=+68.917990712" watchObservedRunningTime="2026-07-22 11:25:00.411688438 +0000 UTC m=+68.919589391" Jul 22 11:25:00.702616 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.702506 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:25:00.705431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:00.705375 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:25:01.387390 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:01.383927 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:25:01.387390 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:01.387041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5568f487dd-hjg9f" Jul 22 11:25:04.783912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:04.783875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:25:04.786271 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:04.786247 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/04ee0ab4-1873-4586-94a1-eb901c612e8c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5z47w\" (UID: \"04ee0ab4-1873-4586-94a1-eb901c612e8c\") " pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:25:05.005575 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.005537 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5pxx7\"" Jul 22 11:25:05.013616 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.013579 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5z47w" Jul 22 11:25:05.146156 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.146125 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5z47w"] Jul 22 11:25:05.149120 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:05.149088 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ee0ab4_1873_4586_94a1_eb901c612e8c.slice/crio-cbad5babb7656470629eed876b9c27843a6474c27877091990a49d49a55deaf2 WatchSource:0}: Error finding container cbad5babb7656470629eed876b9c27843a6474c27877091990a49d49a55deaf2: Status 404 returned error can't find the container with id cbad5babb7656470629eed876b9c27843a6474c27877091990a49d49a55deaf2 Jul 22 11:25:05.406454 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.406412 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z47w" event={"ID":"04ee0ab4-1873-4586-94a1-eb901c612e8c","Type":"ContainerStarted","Data":"cbad5babb7656470629eed876b9c27843a6474c27877091990a49d49a55deaf2"} Jul 22 11:25:05.407969 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.407937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5nfx5" event={"ID":"b184645a-a72b-4102-9927-a276ab76719d","Type":"ContainerStarted","Data":"a032a7c4efaf786e1c922bb8941f81b0b869dcbddb33f28390090489b13c1b71"} Jul 22 11:25:05.409355 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.409326 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" event={"ID":"4fc79d4d-6476-4c68-8ed8-4716cd450a4b","Type":"ContainerStarted","Data":"9622ed4f09450aa12f2c0aa30f0f41ec58568aa48b942771f7c68eb6bba3404c"} Jul 22 11:25:05.410753 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.410734 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dclsj" event={"ID":"6e4ee25f-44d9-430b-ba57-356629e25597","Type":"ContainerStarted","Data":"70f0cf320bdb6a00b9762c5457cf343f2645b3f30d5d773de5ba1cbfc849f37d"} Jul 22 11:25:05.410859 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.410829 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:25:05.412514 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.412490 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzxt" event={"ID":"16572c34-03d5-47fa-9182-fbfc27acf6d5","Type":"ContainerStarted","Data":"0c77fd397952fa2ea670e9381104e25181cd85137b87275856d72b85f6345b85"} Jul 22 11:25:05.412649 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.412542 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pbzxt" event={"ID":"16572c34-03d5-47fa-9182-fbfc27acf6d5","Type":"ContainerStarted","Data":"0c54ec2a7c043dace4c2cb295813c2b7d3aa8c6c8a7f8f97f9b0264d3eb6cf78"} Jul 22 11:25:05.412649 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.412581 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pbzxt" Jul 22 11:25:05.443320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.443272 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5nfx5" podStartSLOduration=67.29738944 podStartE2EDuration="1m13.443256014s" totalImagesPullingTime="6.145866574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:05.425011027 +0000 UTC m=+73.932911913" watchObservedRunningTime="2026-07-22 11:25:05.443256014 +0000 UTC m=+73.951156900" Jul 22 11:25:05.443510 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.443424 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pbzxt" podStartSLOduration=34.455229302 podStartE2EDuration="39.443419932s" totalImagesPullingTime="4.98819063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:24:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:05.442856755 +0000 UTC m=+73.950757640" watchObservedRunningTime="2026-07-22 11:25:05.443419932 +0000 UTC m=+73.951320818" Jul 22 11:25:05.462957 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.462900 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-l92nk" podStartSLOduration=34.061396148 podStartE2EDuration="38.462883822s" totalImagesPullingTime="4.401487674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:05.462092601 +0000 UTC m=+73.969993487" watchObservedRunningTime="2026-07-22 11:25:05.462883822 +0000 UTC m=+73.970784708" Jul 22 11:25:05.477954 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:05.477900 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dclsj" podStartSLOduration=67.368862495 podStartE2EDuration="1m13.477879274s" totalImagesPullingTime="6.109016779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:23:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:05.477153729 +0000 UTC m=+73.985054615" watchObservedRunningTime="2026-07-22 11:25:05.477879274 +0000 UTC m=+73.985780161" Jul 22 11:25:06.609889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.609858 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p"] Jul 22 11:25:06.612700 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.612679 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.616000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.615964 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 11:25:06.616438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.616416 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 11:25:06.616869 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.616842 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-n6ftq\"" Jul 22 11:25:06.616985 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.616909 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 11:25:06.616985 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.616947 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 11:25:06.637866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.637836 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p"] Jul 22 11:25:06.697968 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.697928 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9783c9-bd23-4213-afb9-98521d9817d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.698144 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.697972 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkk7w\" (UniqueName: \"kubernetes.io/projected/ff9783c9-bd23-4213-afb9-98521d9817d9-kube-api-access-nkk7w\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.699718 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.699688 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-584b8fc64-4rjkv"] Jul 22 11:25:06.701768 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.701742 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.704308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.704289 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 11:25:06.704662 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.704645 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7bmns\"" Jul 22 11:25:06.705892 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.705876 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 11:25:06.707148 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.707128 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 11:25:06.712335 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.712313 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 11:25:06.720166 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.720147 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-584b8fc64-4rjkv"] Jul 22 11:25:06.742249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.742222 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-584b8fc64-4rjkv"] Jul 22 11:25:06.742476 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:25:06.742449 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-lzxvw registry-certificates registry-tls trusted-ca], unattached volumes=[], failed to process volumes=[bound-sa-token ca-trust-extracted image-registry-private-configuration installation-pull-secrets kube-api-access-lzxvw registry-certificates registry-tls trusted-ca]: context canceled" pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" podUID="b247f8c8-886a-49f2-9f82-bac3bc233ff0" Jul 22 11:25:06.767943 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.767901 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7875dffb45-85rxj"] Jul 22 11:25:06.770246 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.770218 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.783325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.783289 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7875dffb45-85rxj"] Jul 22 11:25:06.799119 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799080 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799169 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzxvw\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799212 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799450 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799450 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799438 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9783c9-bd23-4213-afb9-98521d9817d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.799558 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799558 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799507 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkk7w\" (UniqueName: \"kubernetes.io/projected/ff9783c9-bd23-4213-afb9-98521d9817d9-kube-api-access-nkk7w\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.799558 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.799695 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.799560 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.802620 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.802566 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff9783c9-bd23-4213-afb9-98521d9817d9-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.818526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.812359 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkk7w\" (UniqueName: \"kubernetes.io/projected/ff9783c9-bd23-4213-afb9-98521d9817d9-kube-api-access-nkk7w\") pod \"managed-serviceaccount-addon-agent-65686b5fbb-6bs4p\" (UID: \"ff9783c9-bd23-4213-afb9-98521d9817d9\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.900919 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.900985 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzxvw\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901017 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-tls\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-certificates\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-trusted-ca\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901106 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9kww\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-kube-api-access-n9kww\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901135 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901178 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-installation-pull-secrets\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901211 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901294 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-bound-sa-token\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901325 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901363 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-image-registry-private-configuration\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901398 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.901889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901649 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a347193c-cf7a-402b-8c5f-2c3752edd5f2-ca-trust-extracted\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:06.902166 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.901947 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.902562 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.902537 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.902945 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.902922 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.905070 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.905037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.905532 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.905508 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.906553 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.906530 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.921980 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.921951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzxvw\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.922201 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.922180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token\") pod \"image-registry-584b8fc64-4rjkv\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:06.932070 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:06.932040 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" Jul 22 11:25:07.002365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-bound-sa-token\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002370 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-image-registry-private-configuration\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002400 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a347193c-cf7a-402b-8c5f-2c3752edd5f2-ca-trust-extracted\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002448 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-tls\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002472 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-certificates\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-trusted-ca\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9kww\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-kube-api-access-n9kww\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.002705 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.002576 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-installation-pull-secrets\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.006727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.006687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-certificates\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.008225 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.005721 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a347193c-cf7a-402b-8c5f-2c3752edd5f2-ca-trust-extracted\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.008225 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.008167 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-installation-pull-secrets\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.013542 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.009632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a347193c-cf7a-402b-8c5f-2c3752edd5f2-trusted-ca\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.013542 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.010902 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-registry-tls\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.013542 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.011661 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a347193c-cf7a-402b-8c5f-2c3752edd5f2-image-registry-private-configuration\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.017220 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.017194 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9kww\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-kube-api-access-n9kww\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.021239 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.021193 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a347193c-cf7a-402b-8c5f-2c3752edd5f2-bound-sa-token\") pod \"image-registry-7875dffb45-85rxj\" (UID: \"a347193c-cf7a-402b-8c5f-2c3752edd5f2\") " pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.081711 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.081674 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:07.104212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.104146 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p"] Jul 22 11:25:07.108833 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:07.108801 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff9783c9_bd23_4213_afb9_98521d9817d9.slice/crio-ee5b2c7856ec8e340e7c6d77f99d6c79bd28f1bf848c0788ff09778a513e5447 WatchSource:0}: Error finding container ee5b2c7856ec8e340e7c6d77f99d6c79bd28f1bf848c0788ff09778a513e5447: Status 404 returned error can't find the container with id ee5b2c7856ec8e340e7c6d77f99d6c79bd28f1bf848c0788ff09778a513e5447 Jul 22 11:25:07.255646 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.255609 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7875dffb45-85rxj"] Jul 22 11:25:07.259028 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:07.258990 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda347193c_cf7a_402b_8c5f_2c3752edd5f2.slice/crio-6ca0d1f632242b98be538f9f8e1d1cd23f1a52183d251116b71aa77c026e76d7 WatchSource:0}: Error finding container 6ca0d1f632242b98be538f9f8e1d1cd23f1a52183d251116b71aa77c026e76d7: Status 404 returned error can't find the container with id 6ca0d1f632242b98be538f9f8e1d1cd23f1a52183d251116b71aa77c026e76d7 Jul 22 11:25:07.421862 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.421820 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" event={"ID":"ff9783c9-bd23-4213-afb9-98521d9817d9","Type":"ContainerStarted","Data":"ee5b2c7856ec8e340e7c6d77f99d6c79bd28f1bf848c0788ff09778a513e5447"} Jul 22 11:25:07.423667 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.423555 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" event={"ID":"a347193c-cf7a-402b-8c5f-2c3752edd5f2","Type":"ContainerStarted","Data":"2cc9c63315191a529bbb7db47d6a9190b08b4d622b935096f1d7eb148d213740"} Jul 22 11:25:07.423667 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.423582 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:07.423667 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.423631 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" event={"ID":"a347193c-cf7a-402b-8c5f-2c3752edd5f2","Type":"ContainerStarted","Data":"6ca0d1f632242b98be538f9f8e1d1cd23f1a52183d251116b71aa77c026e76d7"} Jul 22 11:25:07.430091 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.430067 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:07.464772 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.464662 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" podStartSLOduration=1.464640757 podStartE2EDuration="1.464640757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:07.462850337 +0000 UTC m=+75.970751224" watchObservedRunningTime="2026-07-22 11:25:07.464640757 +0000 UTC m=+75.972541644" Jul 22 11:25:07.507308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507272 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507324 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzxvw\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507365 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507395 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507420 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507484 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507538 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.507817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.507568 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls\") pod \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\" (UID: \"b247f8c8-886a-49f2-9f82-bac3bc233ff0\") " Jul 22 11:25:07.509142 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.508150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:25:07.509314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.509265 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:25:07.509314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.509281 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:25:07.511111 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.511081 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:25:07.511233 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.511132 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:25:07.511233 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.511179 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:25:07.511615 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.511555 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw" (OuterVolumeSpecName: "kube-api-access-lzxvw") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "kube-api-access-lzxvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:25:07.511829 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.511809 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b247f8c8-886a-49f2-9f82-bac3bc233ff0" (UID: "b247f8c8-886a-49f2-9f82-bac3bc233ff0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:25:07.609258 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609220 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-certificates\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609258 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609252 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-registry-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609258 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609263 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b247f8c8-886a-49f2-9f82-bac3bc233ff0-ca-trust-extracted\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609546 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609272 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzxvw\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-kube-api-access-lzxvw\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609546 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609282 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-image-registry-private-configuration\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609546 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609293 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b247f8c8-886a-49f2-9f82-bac3bc233ff0-trusted-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609546 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609306 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b247f8c8-886a-49f2-9f82-bac3bc233ff0-bound-sa-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:07.609546 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:07.609320 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b247f8c8-886a-49f2-9f82-bac3bc233ff0-installation-pull-secrets\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:25:08.428237 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:08.428200 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-584b8fc64-4rjkv" Jul 22 11:25:08.428984 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:08.428914 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:08.464383 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:08.464342 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-584b8fc64-4rjkv"] Jul 22 11:25:08.469238 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:08.469183 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-584b8fc64-4rjkv"] Jul 22 11:25:10.069772 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:10.069357 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b247f8c8-886a-49f2-9f82-bac3bc233ff0" path="/var/lib/kubelet/pods/b247f8c8-886a-49f2-9f82-bac3bc233ff0/volumes" Jul 22 11:25:13.451049 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:13.451009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z47w" event={"ID":"04ee0ab4-1873-4586-94a1-eb901c612e8c","Type":"ContainerStarted","Data":"7e5e0a6e04fb1891c51b147b275fe2130cff5a410de53af0f114350a904f974a"} Jul 22 11:25:13.452789 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:13.452759 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" event={"ID":"ff9783c9-bd23-4213-afb9-98521d9817d9","Type":"ContainerStarted","Data":"ce3041c6b6bc3479d550b2ced854ea685d7a63e23302607a75a313c0b795d32b"} Jul 22 11:25:13.470350 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:13.470298 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-65686b5fbb-6bs4p" podStartSLOduration=1.932189486 podStartE2EDuration="7.470275568s" totalImagesPullingTime="5.538086082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:13.469183059 +0000 UTC m=+81.977083945" watchObservedRunningTime="2026-07-22 11:25:13.470275568 +0000 UTC m=+81.978176451" Jul 22 11:25:15.419184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:15.419142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pbzxt" Jul 22 11:25:16.057345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.057317 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2"] Jul 22 11:25:16.059264 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.059248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.061571 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.061518 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 22 11:25:16.061719 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.061547 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 22 11:25:16.062182 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.062165 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hds5j\"" Jul 22 11:25:16.062266 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.062182 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 11:25:16.077367 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.077347 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2"] Jul 22 11:25:16.170457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.170425 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aa846e3-a096-4097-9631-b69c94d5bdd8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.170645 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.170472 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.170645 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.170493 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wq9k\" (UniqueName: \"kubernetes.io/projected/5aa846e3-a096-4097-9631-b69c94d5bdd8-kube-api-access-9wq9k\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.170645 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.170522 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.271116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.271031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aa846e3-a096-4097-9631-b69c94d5bdd8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.271116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.271078 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.271116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.271098 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wq9k\" (UniqueName: \"kubernetes.io/projected/5aa846e3-a096-4097-9631-b69c94d5bdd8-kube-api-access-9wq9k\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.271358 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.271124 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.271831 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.271804 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aa846e3-a096-4097-9631-b69c94d5bdd8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.273590 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.273561 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.273758 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.273738 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5aa846e3-a096-4097-9631-b69c94d5bdd8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.284636 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.284580 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wq9k\" (UniqueName: \"kubernetes.io/projected/5aa846e3-a096-4097-9631-b69c94d5bdd8-kube-api-access-9wq9k\") pod \"prometheus-operator-5945b5cfd-5k7s2\" (UID: \"5aa846e3-a096-4097-9631-b69c94d5bdd8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.369304 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.369269 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" Jul 22 11:25:16.466833 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.466793 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z47w" event={"ID":"04ee0ab4-1873-4586-94a1-eb901c612e8c","Type":"ContainerStarted","Data":"6a25add47554392b49db984e2b79295df404d7a4e11ff52138d9df8148f170cd"} Jul 22 11:25:16.499425 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:16.499325 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2"] Jul 22 11:25:16.501866 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:16.501846 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa846e3_a096_4097_9631_b69c94d5bdd8.slice/crio-70f6e9be83622e7253830c41cca4a5840414e734e09b10224756c5decde798ae WatchSource:0}: Error finding container 70f6e9be83622e7253830c41cca4a5840414e734e09b10224756c5decde798ae: Status 404 returned error can't find the container with id 70f6e9be83622e7253830c41cca4a5840414e734e09b10224756c5decde798ae Jul 22 11:25:17.473070 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:17.473030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" event={"ID":"5aa846e3-a096-4097-9631-b69c94d5bdd8","Type":"ContainerStarted","Data":"70f6e9be83622e7253830c41cca4a5840414e734e09b10224756c5decde798ae"} Jul 22 11:25:18.320441 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.320364 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:25:18.323610 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.323578 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.326102 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326075 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 11:25:18.326235 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326075 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 11:25:18.326235 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326125 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 11:25:18.327210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326843 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 11:25:18.327210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326865 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sqhjl\"" Jul 22 11:25:18.327210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326877 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 11:25:18.327210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326843 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 11:25:18.327210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.326934 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 11:25:18.351211 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.350520 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:25:18.478760 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.478711 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" event={"ID":"5aa846e3-a096-4097-9631-b69c94d5bdd8","Type":"ContainerStarted","Data":"15fd71dca5c1829e158644e2a7a82f41133c91b71b9e3966fa0ec9be4b5a3480"} Jul 22 11:25:18.479181 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.478815 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" event={"ID":"5aa846e3-a096-4097-9631-b69c94d5bdd8","Type":"ContainerStarted","Data":"d36593415102f353e764ff8f557e80a386d3632d7fda184bbf4c1cb621255ff8"} Jul 22 11:25:18.489848 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.489818 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm2cc\" (UniqueName: \"kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.489994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.489854 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.489994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.489886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.489994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.489933 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.489994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.489965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.490162 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.490055 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.503959 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.503902 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5k7s2" podStartSLOduration=1.051018454 podStartE2EDuration="2.503882913s" totalImagesPullingTime="1.452864459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:18.503267355 +0000 UTC m=+87.011168241" watchObservedRunningTime="2026-07-22 11:25:18.503882913 +0000 UTC m=+87.011783798" Jul 22 11:25:18.590721 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590647 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.590721 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590708 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.590942 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590759 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.590942 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2cc\" (UniqueName: \"kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.590942 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590838 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.590942 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.590906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.591446 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.591414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.591573 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.591508 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.592020 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.591996 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.593979 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.593951 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.593979 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.593975 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.603454 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.603424 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2cc\" (UniqueName: \"kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc\") pod \"console-8c6fdf8b-vhf7v\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:18.661023 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:18.660968 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:20.035366 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.035336 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:25:20.038376 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:20.038349 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706e3c7f_c956_4b99_a695_bea24df1cc5c.slice/crio-e8e010c51cf087182d7d5f12b0e6663fd2ecd5e0e5846ce3a223e2f824811f3d WatchSource:0}: Error finding container e8e010c51cf087182d7d5f12b0e6663fd2ecd5e0e5846ce3a223e2f824811f3d: Status 404 returned error can't find the container with id e8e010c51cf087182d7d5f12b0e6663fd2ecd5e0e5846ce3a223e2f824811f3d Jul 22 11:25:20.488184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.488144 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5z47w" event={"ID":"04ee0ab4-1873-4586-94a1-eb901c612e8c","Type":"ContainerStarted","Data":"b1d957057a572152e7095ee457ea13521810238bc013bc0978b318cfa2113ffc"} Jul 22 11:25:20.489500 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.489465 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c6fdf8b-vhf7v" event={"ID":"706e3c7f-c956-4b99-a695-bea24df1cc5c","Type":"ContainerStarted","Data":"e8e010c51cf087182d7d5f12b0e6663fd2ecd5e0e5846ce3a223e2f824811f3d"} Jul 22 11:25:20.510145 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.510101 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5z47w" podStartSLOduration=33.837421569 podStartE2EDuration="48.510085882s" totalImagesPullingTime="14.672664313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 11:24:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:20.508908593 +0000 UTC m=+89.016809479" watchObservedRunningTime="2026-07-22 11:25:20.510085882 +0000 UTC m=+89.017986769" Jul 22 11:25:20.674536 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.674504 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-d8z4l"] Jul 22 11:25:20.677131 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.677108 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jg98n"] Jul 22 11:25:20.677256 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.677236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.679393 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.679373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.679497 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.679410 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nrfqm\"" Jul 22 11:25:20.679585 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.679562 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 11:25:20.679975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.679958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 11:25:20.681280 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.681261 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 11:25:20.681549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.681530 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 11:25:20.681794 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.681779 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 11:25:20.682007 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.681993 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 11:25:20.684116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.684096 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nfv49\"" Jul 22 11:25:20.693632 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.693582 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-d8z4l"] Jul 22 11:25:20.810838 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.810838 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810797 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-tls\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-metrics-client-ca\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810882 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.810973 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkd5s\" (UniqueName: \"kubernetes.io/projected/2819e0f6-29c5-45bc-a6f2-bde8186c324e-kube-api-access-zkd5s\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811004 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-textfile\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.811066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-root\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811108 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-wtmp\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811141 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-sys\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811197 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811244 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.811431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.811302 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmd5d\" (UniqueName: \"kubernetes.io/projected/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-api-access-cmd5d\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912055 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912015 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-tls\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-metrics-client-ca\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912157 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912249 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkd5s\" (UniqueName: \"kubernetes.io/projected/2819e0f6-29c5-45bc-a6f2-bde8186c324e-kube-api-access-zkd5s\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912504 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912392 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-textfile\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912504 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912504 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912461 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912504 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912493 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-root\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912520 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-wtmp\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912559 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-sys\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912590 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912694 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmd5d\" (UniqueName: \"kubernetes.io/projected/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-api-access-cmd5d\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-textfile\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912809 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-metrics-client-ca\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912815 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912917 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-wtmp\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.912978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.912961 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-root\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.913230 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.913020 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.913230 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.913037 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2819e0f6-29c5-45bc-a6f2-bde8186c324e-sys\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.913696 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.913533 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.915353 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.915332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.915459 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.915393 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.915459 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.915398 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.916517 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.916498 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2819e0f6-29c5-45bc-a6f2-bde8186c324e-node-exporter-tls\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.922281 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.922257 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkd5s\" (UniqueName: \"kubernetes.io/projected/2819e0f6-29c5-45bc-a6f2-bde8186c324e-kube-api-access-zkd5s\") pod \"node-exporter-jg98n\" (UID: \"2819e0f6-29c5-45bc-a6f2-bde8186c324e\") " pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:20.922481 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.922457 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmd5d\" (UniqueName: \"kubernetes.io/projected/5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0-kube-api-access-cmd5d\") pod \"kube-state-metrics-56c5594674-d8z4l\" (UID: \"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.990471 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.990430 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" Jul 22 11:25:20.996325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:20.996298 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jg98n" Jul 22 11:25:21.011055 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:21.011014 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2819e0f6_29c5_45bc_a6f2_bde8186c324e.slice/crio-375883da70d5d7f8bc3fd06f0a3921f7a7fe99a104f8cf3de5220fc53f7cd615 WatchSource:0}: Error finding container 375883da70d5d7f8bc3fd06f0a3921f7a7fe99a104f8cf3de5220fc53f7cd615: Status 404 returned error can't find the container with id 375883da70d5d7f8bc3fd06f0a3921f7a7fe99a104f8cf3de5220fc53f7cd615 Jul 22 11:25:21.162692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:21.162666 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-d8z4l"] Jul 22 11:25:21.167580 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:21.167537 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7f67ff_0ac5_4bb6_9f14_6a5a04f572c0.slice/crio-98cb587f914d3d1f6cc598e0f7a05eadccb57a02a67133417dfaada093a5e880 WatchSource:0}: Error finding container 98cb587f914d3d1f6cc598e0f7a05eadccb57a02a67133417dfaada093a5e880: Status 404 returned error can't find the container with id 98cb587f914d3d1f6cc598e0f7a05eadccb57a02a67133417dfaada093a5e880 Jul 22 11:25:21.495179 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:21.495094 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jg98n" event={"ID":"2819e0f6-29c5-45bc-a6f2-bde8186c324e","Type":"ContainerStarted","Data":"375883da70d5d7f8bc3fd06f0a3921f7a7fe99a104f8cf3de5220fc53f7cd615"} Jul 22 11:25:21.496852 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:21.496819 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" event={"ID":"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0","Type":"ContainerStarted","Data":"98cb587f914d3d1f6cc598e0f7a05eadccb57a02a67133417dfaada093a5e880"} Jul 22 11:25:24.510070 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.510028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c6fdf8b-vhf7v" event={"ID":"706e3c7f-c956-4b99-a695-bea24df1cc5c","Type":"ContainerStarted","Data":"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8"} Jul 22 11:25:24.511588 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.511565 2577 generic.go:356] "Generic (PLEG): container finished" podID="2819e0f6-29c5-45bc-a6f2-bde8186c324e" containerID="dc3faabf1f3d4a50380e0ac8243abafd449cafe0e521a9839a71cf604512216f" exitCode=0 Jul 22 11:25:24.511704 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.511632 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jg98n" event={"ID":"2819e0f6-29c5-45bc-a6f2-bde8186c324e","Type":"ContainerDied","Data":"dc3faabf1f3d4a50380e0ac8243abafd449cafe0e521a9839a71cf604512216f"} Jul 22 11:25:24.513812 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.513783 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" event={"ID":"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0","Type":"ContainerStarted","Data":"6afb3f9fa6c2e258bcbcc3a40fa3c4ca803968dc8e4c6f7171a7f8d8798efa93"} Jul 22 11:25:24.513913 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.513824 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" event={"ID":"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0","Type":"ContainerStarted","Data":"a8040ac3ed2eba8daf2a1185ca23c7a923ee1821596b56a08ab03f67331276eb"} Jul 22 11:25:24.513913 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.513843 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" event={"ID":"5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0","Type":"ContainerStarted","Data":"e8a8d388fae272e66100d9cf6c4c92d4319c8ec7b780bcaafd8e0bb2d61e9219"} Jul 22 11:25:24.532754 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.532711 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8c6fdf8b-vhf7v" podStartSLOduration=2.871280768 podStartE2EDuration="6.532695759s" totalImagesPullingTime="3.661414991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:24.531494228 +0000 UTC m=+93.039395117" watchObservedRunningTime="2026-07-22 11:25:24.532695759 +0000 UTC m=+93.040596646" Jul 22 11:25:24.561912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:24.561857 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-d8z4l" podStartSLOduration=2.036281942 podStartE2EDuration="4.56183668s" totalImagesPullingTime="2.525554738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:24.559847163 +0000 UTC m=+93.067748247" watchObservedRunningTime="2026-07-22 11:25:24.56183668 +0000 UTC m=+93.069737568" Jul 22 11:25:25.262287 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.262247 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p"] Jul 22 11:25:25.266209 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.266183 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.268503 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.268483 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6nzh7\"" Jul 22 11:25:25.268616 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.268575 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 11:25:25.274981 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.274957 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p"] Jul 22 11:25:25.354750 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.354715 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.354750 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.354751 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cnw6\" (UniqueName: \"kubernetes.io/projected/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-kube-api-access-6cnw6\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.455225 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.455190 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.455225 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.455230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cnw6\" (UniqueName: \"kubernetes.io/projected/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-kube-api-access-6cnw6\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.457899 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.457871 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.464931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.464905 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cnw6\" (UniqueName: \"kubernetes.io/projected/3ccb59b6-0c9c-4441-96bc-4197e792fe7b-kube-api-access-6cnw6\") pod \"monitoring-plugin-7b7b7bc8bf-psj7p\" (UID: \"3ccb59b6-0c9c-4441-96bc-4197e792fe7b\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.520336 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.520244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jg98n" event={"ID":"2819e0f6-29c5-45bc-a6f2-bde8186c324e","Type":"ContainerStarted","Data":"c46c00731fe7578f81e227c83bde3685d2e2631443e30a6778c354986d71e2f9"} Jul 22 11:25:25.520336 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.520306 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jg98n" event={"ID":"2819e0f6-29c5-45bc-a6f2-bde8186c324e","Type":"ContainerStarted","Data":"e9b59810c5647b9be08a6d199df0565fa5a958cf14a7945de02e00497fd51f6d"} Jul 22 11:25:25.542414 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.542350 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jg98n" podStartSLOduration=2.763583086 podStartE2EDuration="5.542328394s" totalImagesPullingTime="2.676770741s" totalInitContainerRuntime="101.974567ms" isStatefulPod=true podCreationTimestamp="2026-07-22 11:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:25.541746615 +0000 UTC m=+94.049647539" watchObservedRunningTime="2026-07-22 11:25:25.542328394 +0000 UTC m=+94.050229285" Jul 22 11:25:25.577019 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.576982 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:25.713135 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:25.713098 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p"] Jul 22 11:25:25.719151 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:25:25.719083 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccb59b6_0c9c_4441_96bc_4197e792fe7b.slice/crio-ee9447c1d6846e63003bf3c8a45167f45d50105f7fe5f1866ae6c6722495e99b WatchSource:0}: Error finding container ee9447c1d6846e63003bf3c8a45167f45d50105f7fe5f1866ae6c6722495e99b: Status 404 returned error can't find the container with id ee9447c1d6846e63003bf3c8a45167f45d50105f7fe5f1866ae6c6722495e99b Jul 22 11:25:26.525212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:26.525169 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" event={"ID":"3ccb59b6-0c9c-4441-96bc-4197e792fe7b","Type":"ContainerStarted","Data":"ee9447c1d6846e63003bf3c8a45167f45d50105f7fe5f1866ae6c6722495e99b"} Jul 22 11:25:27.530514 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:27.530428 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" event={"ID":"3ccb59b6-0c9c-4441-96bc-4197e792fe7b","Type":"ContainerStarted","Data":"acfc5b9d81c480f5f8f687715e4f33ce8579347b2e0c239a149fccc616881c10"} Jul 22 11:25:27.530904 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:27.530632 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:27.535391 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:27.535369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" Jul 22 11:25:27.548825 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:27.548775 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-psj7p" podStartSLOduration=1.09996707 podStartE2EDuration="2.548759093s" totalImagesPullingTime="1.448792023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:25:27.548211099 +0000 UTC m=+96.056111985" watchObservedRunningTime="2026-07-22 11:25:27.548759093 +0000 UTC m=+96.056659979" Jul 22 11:25:28.662222 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:28.662188 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:28.662725 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:28.662255 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:28.666941 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:28.666919 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:29.437806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:29.437776 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7875dffb45-85rxj" Jul 22 11:25:29.541503 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:29.541475 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:25:36.419184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:36.419155 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dclsj" Jul 22 11:25:39.994529 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:25:39.994493 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:26:05.015493 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.015423 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8c6fdf8b-vhf7v" podUID="706e3c7f-c956-4b99-a695-bea24df1cc5c" containerName="console" containerID="cri-o://050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8" gracePeriod=15 Jul 22 11:26:05.261607 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.261569 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:26:05.352764 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352677 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zm2cc\" (UniqueName: \"kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.352764 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352725 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.352764 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352758 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.353018 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352778 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.353018 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352813 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.353018 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.352859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert\") pod \"706e3c7f-c956-4b99-a695-bea24df1cc5c\" (UID: \"706e3c7f-c956-4b99-a695-bea24df1cc5c\") " Jul 22 11:26:05.353308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.353270 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config" (OuterVolumeSpecName: "console-config") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:26:05.353308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.353277 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca" (OuterVolumeSpecName: "service-ca") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:26:05.353470 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.353280 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:26:05.355206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.355179 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc" (OuterVolumeSpecName: "kube-api-access-zm2cc") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "kube-api-access-zm2cc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:26:05.355206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.355191 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:26:05.355321 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.355302 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "706e3c7f-c956-4b99-a695-bea24df1cc5c" (UID: "706e3c7f-c956-4b99-a695-bea24df1cc5c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:26:05.454315 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454284 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.454315 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454313 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.454528 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454327 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zm2cc\" (UniqueName: \"kubernetes.io/projected/706e3c7f-c956-4b99-a695-bea24df1cc5c-kube-api-access-zm2cc\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.454528 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454341 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.454528 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454351 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/706e3c7f-c956-4b99-a695-bea24df1cc5c-console-oauth-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.454528 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.454362 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/706e3c7f-c956-4b99-a695-bea24df1cc5c-oauth-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:26:05.665901 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.665867 2577 generic.go:356] "Generic (PLEG): container finished" podID="706e3c7f-c956-4b99-a695-bea24df1cc5c" containerID="050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8" exitCode=0 Jul 22 11:26:05.666061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.665910 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c6fdf8b-vhf7v" event={"ID":"706e3c7f-c956-4b99-a695-bea24df1cc5c","Type":"ContainerDied","Data":"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8"} Jul 22 11:26:05.666061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.665916 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8c6fdf8b-vhf7v" Jul 22 11:26:05.666061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.665947 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8c6fdf8b-vhf7v" event={"ID":"706e3c7f-c956-4b99-a695-bea24df1cc5c","Type":"ContainerDied","Data":"e8e010c51cf087182d7d5f12b0e6663fd2ecd5e0e5846ce3a223e2f824811f3d"} Jul 22 11:26:05.666061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.665966 2577 scope.go:117] "RemoveContainer" containerID="050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8" Jul 22 11:26:05.674899 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.674880 2577 scope.go:117] "RemoveContainer" containerID="050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8" Jul 22 11:26:05.675177 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:26:05.675155 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8\": container with ID starting with 050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8 not found: ID does not exist" containerID="050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8" Jul 22 11:26:05.675234 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.675185 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8"} err="failed to get container status \"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8\": rpc error: code = NotFound desc = could not find container \"050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8\": container with ID starting with 050a33001211c5653d2434f6012efa6bcb2e402736af56086a927b203bc5e1a8 not found: ID does not exist" Jul 22 11:26:05.695991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.695962 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:26:05.719083 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:05.719048 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8c6fdf8b-vhf7v"] Jul 22 11:26:06.066323 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:06.066246 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="706e3c7f-c956-4b99-a695-bea24df1cc5c" path="/var/lib/kubelet/pods/706e3c7f-c956-4b99-a695-bea24df1cc5c/volumes" Jul 22 11:26:45.086409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.085019 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9c675b759-6kk74"] Jul 22 11:26:45.086409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.085450 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706e3c7f-c956-4b99-a695-bea24df1cc5c" containerName="console" Jul 22 11:26:45.086409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.085470 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="706e3c7f-c956-4b99-a695-bea24df1cc5c" containerName="console" Jul 22 11:26:45.086409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.085531 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="706e3c7f-c956-4b99-a695-bea24df1cc5c" containerName="console" Jul 22 11:26:45.088997 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.088975 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.092288 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.091710 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-54zdh\"" Jul 22 11:26:45.092288 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.091945 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 11:26:45.092288 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.092152 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 11:26:45.094663 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.094623 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 11:26:45.094663 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.094636 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 11:26:45.094895 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.094880 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 11:26:45.115995 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.115961 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9c675b759-6kk74"] Jul 22 11:26:45.130039 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.130013 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 11:26:45.256423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256432 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-federate-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5l7b\" (UniqueName: \"kubernetes.io/projected/c38c02c7-6666-4002-8b41-1b7a8e5d8215-kube-api-access-d5l7b\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256548 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-serving-certs-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.256665 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.256646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-metrics-client-ca\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.357894 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.357854 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-metrics-client-ca\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.357940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.357972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-federate-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5l7b\" (UniqueName: \"kubernetes.io/projected/c38c02c7-6666-4002-8b41-1b7a8e5d8215-kube-api-access-d5l7b\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358414 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358116 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-serving-certs-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.358845 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358776 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-metrics-client-ca\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.359144 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.358993 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-serving-certs-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.359288 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.359263 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.360861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.360829 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.360966 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.360835 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-telemeter-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.360966 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.360903 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-secret-telemeter-client\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.360966 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.360931 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c38c02c7-6666-4002-8b41-1b7a8e5d8215-federate-client-tls\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.366797 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.366778 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5l7b\" (UniqueName: \"kubernetes.io/projected/c38c02c7-6666-4002-8b41-1b7a8e5d8215-kube-api-access-d5l7b\") pod \"telemeter-client-9c675b759-6kk74\" (UID: \"c38c02c7-6666-4002-8b41-1b7a8e5d8215\") " pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.399861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.399652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" Jul 22 11:26:45.563555 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.563514 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9c675b759-6kk74"] Jul 22 11:26:45.566859 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:26:45.566831 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc38c02c7_6666_4002_8b41_1b7a8e5d8215.slice/crio-775e0c9d826a0381e1497b59bfaadf505317c34db36d66a37f11207dbe061864 WatchSource:0}: Error finding container 775e0c9d826a0381e1497b59bfaadf505317c34db36d66a37f11207dbe061864: Status 404 returned error can't find the container with id 775e0c9d826a0381e1497b59bfaadf505317c34db36d66a37f11207dbe061864 Jul 22 11:26:45.799328 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:45.799243 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" event={"ID":"c38c02c7-6666-4002-8b41-1b7a8e5d8215","Type":"ContainerStarted","Data":"775e0c9d826a0381e1497b59bfaadf505317c34db36d66a37f11207dbe061864"} Jul 22 11:26:47.807967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:47.807933 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" event={"ID":"c38c02c7-6666-4002-8b41-1b7a8e5d8215","Type":"ContainerStarted","Data":"3e26683bd853445d3bb260a187b5d2fe5fc1acc46239799d669fcefe888c2c80"} Jul 22 11:26:49.816890 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:49.816854 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" event={"ID":"c38c02c7-6666-4002-8b41-1b7a8e5d8215","Type":"ContainerStarted","Data":"2aa6775ce6523892c50ca1add0c5172eabe845c4c05a9aab91f8074f38d2a203"} Jul 22 11:26:49.817383 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:49.816916 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" event={"ID":"c38c02c7-6666-4002-8b41-1b7a8e5d8215","Type":"ContainerStarted","Data":"cd2a391d0af5e72aa0b22b49ba3c5e2820b5dfd1b3b2a90846d9682fc889b5b2"} Jul 22 11:26:49.841842 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:49.841787 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9c675b759-6kk74" podStartSLOduration=1.607636088 podStartE2EDuration="4.841769825s" totalImagesPullingTime="3.234133737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:26:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:26:49.840288415 +0000 UTC m=+178.348189301" watchObservedRunningTime="2026-07-22 11:26:49.841769825 +0000 UTC m=+178.349670710" Jul 22 11:26:50.543146 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.543112 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:26:50.546335 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.546315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.549309 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.549281 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 11:26:50.549428 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.549285 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 11:26:50.550234 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550208 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 11:26:50.550312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550288 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 11:26:50.550367 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550339 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sqhjl\"" Jul 22 11:26:50.550533 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550517 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 11:26:50.550628 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550574 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 11:26:50.550695 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.550658 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 11:26:50.559090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.559068 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 11:26:50.561692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.561671 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:26:50.598409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598427 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kl7g\" (UniqueName: \"kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598478 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598516 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.598647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.598553 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.699824 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.699782 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.699824 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.699823 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700076 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.699844 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700076 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.699963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700177 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700081 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700177 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700177 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700147 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kl7g\" (UniqueName: \"kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700676 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700648 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700851 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700899 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700861 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.700990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.700974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.702521 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.702489 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.702637 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.702498 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.709072 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.709052 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kl7g\" (UniqueName: \"kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g\") pod \"console-5499f6d56d-rgs6w\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.858404 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.858370 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:26:50.995379 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:50.995355 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:26:50.997862 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:26:50.997833 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod755eb469_f2d4_4e3b_826b_a83e93671eb9.slice/crio-e1c3a533d439d8c3ab59ced6c6d44dd79e6ad96f013455f65fc33ad09f6d4dfa WatchSource:0}: Error finding container e1c3a533d439d8c3ab59ced6c6d44dd79e6ad96f013455f65fc33ad09f6d4dfa: Status 404 returned error can't find the container with id e1c3a533d439d8c3ab59ced6c6d44dd79e6ad96f013455f65fc33ad09f6d4dfa Jul 22 11:26:51.825792 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:51.825757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5499f6d56d-rgs6w" event={"ID":"755eb469-f2d4-4e3b-826b-a83e93671eb9","Type":"ContainerStarted","Data":"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8"} Jul 22 11:26:51.826007 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:51.825803 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5499f6d56d-rgs6w" event={"ID":"755eb469-f2d4-4e3b-826b-a83e93671eb9","Type":"ContainerStarted","Data":"e1c3a533d439d8c3ab59ced6c6d44dd79e6ad96f013455f65fc33ad09f6d4dfa"} Jul 22 11:26:51.845763 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:26:51.845716 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5499f6d56d-rgs6w" podStartSLOduration=1.845699394 podStartE2EDuration="1.845699394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:26:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:26:51.845115028 +0000 UTC m=+180.353015915" watchObservedRunningTime="2026-07-22 11:26:51.845699394 +0000 UTC m=+180.353600279" Jul 22 11:27:00.858726 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:27:00.858690 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:27:00.859114 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:27:00.858750 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:27:00.863663 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:27:00.863635 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:27:01.867937 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:27:01.867907 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:28:17.575562 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:17.575523 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:28:42.596130 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.596082 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5499f6d56d-rgs6w" podUID="755eb469-f2d4-4e3b-826b-a83e93671eb9" containerName="console" containerID="cri-o://09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8" gracePeriod=15 Jul 22 11:28:42.848102 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.848045 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:28:42.995329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995299 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995330 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995376 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kl7g\" (UniqueName: \"kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995403 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995480 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995769 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995562 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config\") pod \"755eb469-f2d4-4e3b-826b-a83e93671eb9\" (UID: \"755eb469-f2d4-4e3b-826b-a83e93671eb9\") " Jul 22 11:28:42.995861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995815 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config" (OuterVolumeSpecName: "console-config") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:28:42.995861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995821 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:28:42.996019 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.995997 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:28:42.996077 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.996053 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca" (OuterVolumeSpecName: "service-ca") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:28:42.997915 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.997889 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:28:42.998294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.998267 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:28:42.998294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:42.998285 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g" (OuterVolumeSpecName: "kube-api-access-2kl7g") pod "755eb469-f2d4-4e3b-826b-a83e93671eb9" (UID: "755eb469-f2d4-4e3b-826b-a83e93671eb9"). InnerVolumeSpecName "kube-api-access-2kl7g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:28:43.097000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.096949 2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-oauth-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.096995 2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-oauth-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.097007 2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-config\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097000 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.097016 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kl7g\" (UniqueName: \"kubernetes.io/projected/755eb469-f2d4-4e3b-826b-a83e93671eb9-kube-api-access-2kl7g\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.097025 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-trusted-ca-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.097034 2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/755eb469-f2d4-4e3b-826b-a83e93671eb9-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.097247 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.097042 2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/755eb469-f2d4-4e3b-826b-a83e93671eb9-console-serving-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:28:43.221116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.221079 2577 generic.go:356] "Generic (PLEG): container finished" podID="755eb469-f2d4-4e3b-826b-a83e93671eb9" containerID="09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8" exitCode=0 Jul 22 11:28:43.221265 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.221132 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5499f6d56d-rgs6w" Jul 22 11:28:43.221265 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.221163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5499f6d56d-rgs6w" event={"ID":"755eb469-f2d4-4e3b-826b-a83e93671eb9","Type":"ContainerDied","Data":"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8"} Jul 22 11:28:43.221265 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.221217 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5499f6d56d-rgs6w" event={"ID":"755eb469-f2d4-4e3b-826b-a83e93671eb9","Type":"ContainerDied","Data":"e1c3a533d439d8c3ab59ced6c6d44dd79e6ad96f013455f65fc33ad09f6d4dfa"} Jul 22 11:28:43.221265 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.221237 2577 scope.go:117] "RemoveContainer" containerID="09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8" Jul 22 11:28:43.229672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.229655 2577 scope.go:117] "RemoveContainer" containerID="09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8" Jul 22 11:28:43.229946 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:28:43.229927 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8\": container with ID starting with 09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8 not found: ID does not exist" containerID="09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8" Jul 22 11:28:43.229997 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.229955 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8"} err="failed to get container status \"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8\": rpc error: code = NotFound desc = could not find container \"09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8\": container with ID starting with 09685274e4d7fbdd923de9ccadf71868d631150c8729ae88181b09dbaa4fe1d8 not found: ID does not exist" Jul 22 11:28:43.243340 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.243315 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:28:43.248797 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:43.248771 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5499f6d56d-rgs6w"] Jul 22 11:28:44.067667 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:44.067626 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="755eb469-f2d4-4e3b-826b-a83e93671eb9" path="/var/lib/kubelet/pods/755eb469-f2d4-4e3b-826b-a83e93671eb9/volumes" Jul 22 11:28:51.893115 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:51.893078 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:28:51.899142 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:51.899117 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:28:51.899755 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:28:51.899731 2577 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 11:29:03.276476 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.276441 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n"] Jul 22 11:29:03.278785 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.276724 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="755eb469-f2d4-4e3b-826b-a83e93671eb9" containerName="console" Jul 22 11:29:03.278785 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.276736 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="755eb469-f2d4-4e3b-826b-a83e93671eb9" containerName="console" Jul 22 11:29:03.278785 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.276783 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="755eb469-f2d4-4e3b-826b-a83e93671eb9" containerName="console" Jul 22 11:29:03.279645 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.279629 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.282205 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.282182 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:29:03.282806 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.282789 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv7sm\"" Jul 22 11:29:03.282898 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.282805 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:29:03.293082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.293060 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n"] Jul 22 11:29:03.339868 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.339830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.339868 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.339863 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.340087 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.339884 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h9rf\" (UniqueName: \"kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.441339 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.441296 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.441339 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.441334 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.441580 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.441354 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h9rf\" (UniqueName: \"kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.441774 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.441752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.441845 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.441811 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.451021 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.450994 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h9rf\" (UniqueName: \"kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.590751 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.590663 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:03.727108 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.727013 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n"] Jul 22 11:29:03.729866 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:29:03.729837 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1c2953_9794_402f_ae37_f73cac501f24.slice/crio-19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93 WatchSource:0}: Error finding container 19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93: Status 404 returned error can't find the container with id 19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93 Jul 22 11:29:03.731820 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:03.731802 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:29:04.306433 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:04.306398 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" event={"ID":"7f1c2953-9794-402f-ae37-f73cac501f24","Type":"ContainerStarted","Data":"19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93"} Jul 22 11:29:10.337863 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:10.337827 2577 generic.go:356] "Generic (PLEG): container finished" podID="7f1c2953-9794-402f-ae37-f73cac501f24" containerID="599336aacfaf19ccfc784142c3a06adb398d1350a9caf9b4f27f479cd6240c76" exitCode=0 Jul 22 11:29:10.338361 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:10.337888 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" event={"ID":"7f1c2953-9794-402f-ae37-f73cac501f24","Type":"ContainerDied","Data":"599336aacfaf19ccfc784142c3a06adb398d1350a9caf9b4f27f479cd6240c76"} Jul 22 11:29:13.356649 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:13.356585 2577 generic.go:356] "Generic (PLEG): container finished" podID="7f1c2953-9794-402f-ae37-f73cac501f24" containerID="c2413f1f54d5409ed9e1758002cf11e805c7ad546a90afa6d7a7c737d9e3ff67" exitCode=0 Jul 22 11:29:13.357058 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:13.356685 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" event={"ID":"7f1c2953-9794-402f-ae37-f73cac501f24","Type":"ContainerDied","Data":"c2413f1f54d5409ed9e1758002cf11e805c7ad546a90afa6d7a7c737d9e3ff67"} Jul 22 11:29:21.390925 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:21.390888 2577 generic.go:356] "Generic (PLEG): container finished" podID="7f1c2953-9794-402f-ae37-f73cac501f24" containerID="84b725d0b6adc952fa6ca8eb55484210eaebd594f332fd43940fa0fc7de9e7a0" exitCode=0 Jul 22 11:29:21.391325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:21.390973 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" event={"ID":"7f1c2953-9794-402f-ae37-f73cac501f24","Type":"ContainerDied","Data":"84b725d0b6adc952fa6ca8eb55484210eaebd594f332fd43940fa0fc7de9e7a0"} Jul 22 11:29:22.520065 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.520044 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:22.606990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.606960 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h9rf\" (UniqueName: \"kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf\") pod \"7f1c2953-9794-402f-ae37-f73cac501f24\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " Jul 22 11:29:22.606990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.606997 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util\") pod \"7f1c2953-9794-402f-ae37-f73cac501f24\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " Jul 22 11:29:22.607227 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.607035 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle\") pod \"7f1c2953-9794-402f-ae37-f73cac501f24\" (UID: \"7f1c2953-9794-402f-ae37-f73cac501f24\") " Jul 22 11:29:22.607714 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.607687 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle" (OuterVolumeSpecName: "bundle") pod "7f1c2953-9794-402f-ae37-f73cac501f24" (UID: "7f1c2953-9794-402f-ae37-f73cac501f24"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:22.609286 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.609261 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf" (OuterVolumeSpecName: "kube-api-access-2h9rf") pod "7f1c2953-9794-402f-ae37-f73cac501f24" (UID: "7f1c2953-9794-402f-ae37-f73cac501f24"). InnerVolumeSpecName "kube-api-access-2h9rf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:29:22.611802 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.611776 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util" (OuterVolumeSpecName: "util") pod "7f1c2953-9794-402f-ae37-f73cac501f24" (UID: "7f1c2953-9794-402f-ae37-f73cac501f24"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:22.707648 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.707549 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h9rf\" (UniqueName: \"kubernetes.io/projected/7f1c2953-9794-402f-ae37-f73cac501f24-kube-api-access-2h9rf\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:22.707648 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.707576 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:22.707648 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:22.707585 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7f1c2953-9794-402f-ae37-f73cac501f24-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:23.400002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:23.399971 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" event={"ID":"7f1c2953-9794-402f-ae37-f73cac501f24","Type":"ContainerDied","Data":"19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93"} Jul 22 11:29:23.400002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:23.400007 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zvn8n" Jul 22 11:29:23.400282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:23.400017 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19954f01cf4a6decd25ba465154d8f1abc5c8b229540ca0e8071786e073b4a93" Jul 22 11:29:26.556268 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556236 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm"] Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556511 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="pull" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556521 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="pull" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556539 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="util" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556544 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="util" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556551 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="extract" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556557 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="extract" Jul 22 11:29:26.556666 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.556625 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f1c2953-9794-402f-ae37-f73cac501f24" containerName="extract" Jul 22 11:29:26.596493 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.596461 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm"] Jul 22 11:29:26.596677 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.596580 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.599188 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.599158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 11:29:26.599188 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.599170 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-gz5hr\"" Jul 22 11:29:26.599409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.599396 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:29:26.744177 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.744139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdr4m\" (UniqueName: \"kubernetes.io/projected/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-kube-api-access-zdr4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.744359 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.744201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.844937 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.844857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.845066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.844947 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdr4m\" (UniqueName: \"kubernetes.io/projected/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-kube-api-access-zdr4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.845261 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.845231 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.854237 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.854203 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdr4m\" (UniqueName: \"kubernetes.io/projected/031f6012-dcb1-406b-9ab4-365d1ec5c7e4-kube-api-access-zdr4m\") pod \"cert-manager-operator-controller-manager-55c444bf86-5p9nm\" (UID: \"031f6012-dcb1-406b-9ab4-365d1ec5c7e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:26.907759 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:26.907717 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" Jul 22 11:29:27.075726 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:27.075691 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm"] Jul 22 11:29:27.080032 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:29:27.079999 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod031f6012_dcb1_406b_9ab4_365d1ec5c7e4.slice/crio-6f09ac899d5ee30877b56872a6503e093e09401c93165e633191d8b2945b233c WatchSource:0}: Error finding container 6f09ac899d5ee30877b56872a6503e093e09401c93165e633191d8b2945b233c: Status 404 returned error can't find the container with id 6f09ac899d5ee30877b56872a6503e093e09401c93165e633191d8b2945b233c Jul 22 11:29:27.416463 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:27.416421 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" event={"ID":"031f6012-dcb1-406b-9ab4-365d1ec5c7e4","Type":"ContainerStarted","Data":"6f09ac899d5ee30877b56872a6503e093e09401c93165e633191d8b2945b233c"} Jul 22 11:29:29.427218 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:29.427181 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" event={"ID":"031f6012-dcb1-406b-9ab4-365d1ec5c7e4","Type":"ContainerStarted","Data":"95255ddca46acdfc23e03448800ecb0b0e6f9ebfa9aea0e5ed96245ce4e670c5"} Jul 22 11:29:29.456055 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:29.455990 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5p9nm" podStartSLOduration=1.7119456450000001 podStartE2EDuration="3.45597457s" totalImagesPullingTime="1.744028925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:29:29.454952366 +0000 UTC m=+337.962853253" watchObservedRunningTime="2026-07-22 11:29:29.45597457 +0000 UTC m=+337.963875455" Jul 22 11:29:30.633633 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.633581 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq"] Jul 22 11:29:30.635952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.635935 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.638488 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.638469 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv7sm\"" Jul 22 11:29:30.639122 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.639107 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:29:30.639189 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.639139 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:29:30.647917 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.647890 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq"] Jul 22 11:29:30.777967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.777918 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.777967 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.777974 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.778203 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.778038 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx26v\" (UniqueName: \"kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.879300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.879264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.879300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.879303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.879502 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.879336 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx26v\" (UniqueName: \"kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.879783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.879763 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.879828 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.879775 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.896082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.896006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx26v\" (UniqueName: \"kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.944563 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.944530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:30.976696 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.976660 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kspqc"] Jul 22 11:29:30.979057 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.979038 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:30.984821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.984800 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7b6k6\"" Jul 22 11:29:30.985939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.985919 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 11:29:30.986958 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.986940 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 11:29:30.999372 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:30.999350 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kspqc"] Jul 22 11:29:31.080798 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.080761 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcnhw\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-kube-api-access-hcnhw\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.080952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.080814 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.081756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.081586 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq"] Jul 22 11:29:31.084539 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:29:31.084504 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53f11859_62f9_470b_a7af_d99e98bd7622.slice/crio-534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94 WatchSource:0}: Error finding container 534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94: Status 404 returned error can't find the container with id 534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94 Jul 22 11:29:31.182060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.182016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcnhw\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-kube-api-access-hcnhw\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.182201 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.182069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.193883 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.193855 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.194243 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.194221 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcnhw\" (UniqueName: \"kubernetes.io/projected/9cfef7aa-eb11-4c89-9646-545d3a64c8ca-kube-api-access-hcnhw\") pod \"cert-manager-webhook-6486ffd997-kspqc\" (UID: \"9cfef7aa-eb11-4c89-9646-545d3a64c8ca\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.289212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.289182 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:31.422658 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.422621 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kspqc"] Jul 22 11:29:31.425273 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:29:31.425247 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cfef7aa_eb11_4c89_9646_545d3a64c8ca.slice/crio-0abfe7b1caa21aa0834c92f47dfbd4dabf14e0a573169151721ef98ede45b3a1 WatchSource:0}: Error finding container 0abfe7b1caa21aa0834c92f47dfbd4dabf14e0a573169151721ef98ede45b3a1: Status 404 returned error can't find the container with id 0abfe7b1caa21aa0834c92f47dfbd4dabf14e0a573169151721ef98ede45b3a1 Jul 22 11:29:31.436800 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.436774 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" event={"ID":"9cfef7aa-eb11-4c89-9646-545d3a64c8ca","Type":"ContainerStarted","Data":"0abfe7b1caa21aa0834c92f47dfbd4dabf14e0a573169151721ef98ede45b3a1"} Jul 22 11:29:31.438186 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.438159 2577 generic.go:356] "Generic (PLEG): container finished" podID="53f11859-62f9-470b-a7af-d99e98bd7622" containerID="913bcc45b32d05d63165f95ded7cd16f674e68ec207939f3667122da606d2529" exitCode=0 Jul 22 11:29:31.438275 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.438215 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerDied","Data":"913bcc45b32d05d63165f95ded7cd16f674e68ec207939f3667122da606d2529"} Jul 22 11:29:31.438316 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:31.438288 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerStarted","Data":"534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94"} Jul 22 11:29:34.456256 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:34.456219 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerStarted","Data":"27de642e550ff205e6d9cd4e6c67b3367ddd0349587cb0c03da89ed44460544c"} Jul 22 11:29:35.461396 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:35.461352 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" event={"ID":"9cfef7aa-eb11-4c89-9646-545d3a64c8ca","Type":"ContainerStarted","Data":"2aeb0cd41175eaacd9cc8bf0d441941908085d003e56c693e452aa679745460f"} Jul 22 11:29:35.461912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:35.461480 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:35.463117 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:35.463088 2577 generic.go:356] "Generic (PLEG): container finished" podID="53f11859-62f9-470b-a7af-d99e98bd7622" containerID="27de642e550ff205e6d9cd4e6c67b3367ddd0349587cb0c03da89ed44460544c" exitCode=0 Jul 22 11:29:35.463244 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:35.463168 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerDied","Data":"27de642e550ff205e6d9cd4e6c67b3367ddd0349587cb0c03da89ed44460544c"} Jul 22 11:29:35.493946 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:35.493891 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" podStartSLOduration=2.300850827 podStartE2EDuration="5.493871873s" totalImagesPullingTime="3.193021046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:29:35.49354808 +0000 UTC m=+344.001448965" watchObservedRunningTime="2026-07-22 11:29:35.493871873 +0000 UTC m=+344.001772760" Jul 22 11:29:36.472071 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:36.472023 2577 generic.go:356] "Generic (PLEG): container finished" podID="53f11859-62f9-470b-a7af-d99e98bd7622" containerID="886a1c49d7d72010e2636d23dc15f1c89eeac10b9dd062ee4f50b64ea5b6704e" exitCode=0 Jul 22 11:29:36.472530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:36.472122 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerDied","Data":"886a1c49d7d72010e2636d23dc15f1c89eeac10b9dd062ee4f50b64ea5b6704e"} Jul 22 11:29:37.597417 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.597394 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:37.742207 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.742116 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx26v\" (UniqueName: \"kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v\") pod \"53f11859-62f9-470b-a7af-d99e98bd7622\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " Jul 22 11:29:37.742207 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.742155 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util\") pod \"53f11859-62f9-470b-a7af-d99e98bd7622\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " Jul 22 11:29:37.742207 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.742203 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle\") pod \"53f11859-62f9-470b-a7af-d99e98bd7622\" (UID: \"53f11859-62f9-470b-a7af-d99e98bd7622\") " Jul 22 11:29:37.742639 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.742587 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle" (OuterVolumeSpecName: "bundle") pod "53f11859-62f9-470b-a7af-d99e98bd7622" (UID: "53f11859-62f9-470b-a7af-d99e98bd7622"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:37.744412 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.744377 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v" (OuterVolumeSpecName: "kube-api-access-gx26v") pod "53f11859-62f9-470b-a7af-d99e98bd7622" (UID: "53f11859-62f9-470b-a7af-d99e98bd7622"). InnerVolumeSpecName "kube-api-access-gx26v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:29:37.746680 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.746645 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util" (OuterVolumeSpecName: "util") pod "53f11859-62f9-470b-a7af-d99e98bd7622" (UID: "53f11859-62f9-470b-a7af-d99e98bd7622"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:37.843653 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.843584 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:37.843653 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.843648 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx26v\" (UniqueName: \"kubernetes.io/projected/53f11859-62f9-470b-a7af-d99e98bd7622-kube-api-access-gx26v\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:37.843653 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:37.843659 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53f11859-62f9-470b-a7af-d99e98bd7622-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:38.481927 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:38.481896 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" Jul 22 11:29:38.482073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:38.481927 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fv7kxq" event={"ID":"53f11859-62f9-470b-a7af-d99e98bd7622","Type":"ContainerDied","Data":"534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94"} Jul 22 11:29:38.482073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:38.481970 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="534ad38734a975d5552a03ca1f95f5ccb037591642b814aa0a781b7050ddbe94" Jul 22 11:29:41.474371 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:41.474340 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kspqc" Jul 22 11:29:50.144130 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144035 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk"] Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144324 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="extract" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144336 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="extract" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144354 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="pull" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144359 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="pull" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144369 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="util" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144374 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="util" Jul 22 11:29:50.144465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.144417 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="53f11859-62f9-470b-a7af-d99e98bd7622" containerName="extract" Jul 22 11:29:50.147340 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.147323 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.149331 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.149312 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 11:29:50.149804 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.149789 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 11:29:50.149804 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.149797 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-xv7sm\"" Jul 22 11:29:50.155431 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.155406 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk"] Jul 22 11:29:50.240863 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.240832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.241046 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.240869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq2cl\" (UniqueName: \"kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.241046 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.240997 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.342242 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.342194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.342242 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.342247 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.342471 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.342267 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq2cl\" (UniqueName: \"kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.342714 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.342689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.342756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.342697 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.353652 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.353626 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq2cl\" (UniqueName: \"kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.456589 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.456495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:50.590140 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:50.590096 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk"] Jul 22 11:29:50.592510 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:29:50.592478 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf28a6e03_a257_4661_9ec2_ff3d75f025ae.slice/crio-765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab WatchSource:0}: Error finding container 765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab: Status 404 returned error can't find the container with id 765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab Jul 22 11:29:51.539893 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:51.539859 2577 generic.go:356] "Generic (PLEG): container finished" podID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerID="b1cc511b59673a2cad9980b7a3fe7da7969a5c3cf30fd51b869a9c9f48a57b2d" exitCode=0 Jul 22 11:29:51.540334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:51.539946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" event={"ID":"f28a6e03-a257-4661-9ec2-ff3d75f025ae","Type":"ContainerDied","Data":"b1cc511b59673a2cad9980b7a3fe7da7969a5c3cf30fd51b869a9c9f48a57b2d"} Jul 22 11:29:51.540334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:51.539994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" event={"ID":"f28a6e03-a257-4661-9ec2-ff3d75f025ae","Type":"ContainerStarted","Data":"765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab"} Jul 22 11:29:52.546308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:52.546275 2577 generic.go:356] "Generic (PLEG): container finished" podID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerID="d091dc28c1bc6a2049e6bd0499d2970c66fd89950007e64a14d85d9a8844adb4" exitCode=0 Jul 22 11:29:52.546308 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:52.546320 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" event={"ID":"f28a6e03-a257-4661-9ec2-ff3d75f025ae","Type":"ContainerDied","Data":"d091dc28c1bc6a2049e6bd0499d2970c66fd89950007e64a14d85d9a8844adb4"} Jul 22 11:29:53.552690 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:53.552659 2577 generic.go:356] "Generic (PLEG): container finished" podID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerID="f77764fc4355ba193ecff2f6d2876411420bd093863485423350b428bc55a5ce" exitCode=0 Jul 22 11:29:53.553079 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:53.552747 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" event={"ID":"f28a6e03-a257-4661-9ec2-ff3d75f025ae","Type":"ContainerDied","Data":"f77764fc4355ba193ecff2f6d2876411420bd093863485423350b428bc55a5ce"} Jul 22 11:29:54.701841 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.701814 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:54.778132 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.778097 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util\") pod \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " Jul 22 11:29:54.778294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.778152 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle\") pod \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " Jul 22 11:29:54.778294 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.778184 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rq2cl\" (UniqueName: \"kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl\") pod \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\" (UID: \"f28a6e03-a257-4661-9ec2-ff3d75f025ae\") " Jul 22 11:29:54.779091 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.779059 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle" (OuterVolumeSpecName: "bundle") pod "f28a6e03-a257-4661-9ec2-ff3d75f025ae" (UID: "f28a6e03-a257-4661-9ec2-ff3d75f025ae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:54.780379 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.780357 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl" (OuterVolumeSpecName: "kube-api-access-rq2cl") pod "f28a6e03-a257-4661-9ec2-ff3d75f025ae" (UID: "f28a6e03-a257-4661-9ec2-ff3d75f025ae"). InnerVolumeSpecName "kube-api-access-rq2cl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:29:54.784307 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.784152 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util" (OuterVolumeSpecName: "util") pod "f28a6e03-a257-4661-9ec2-ff3d75f025ae" (UID: "f28a6e03-a257-4661-9ec2-ff3d75f025ae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:29:54.879541 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.879507 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rq2cl\" (UniqueName: \"kubernetes.io/projected/f28a6e03-a257-4661-9ec2-ff3d75f025ae-kube-api-access-rq2cl\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:54.879541 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.879536 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:54.879751 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:54.879548 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f28a6e03-a257-4661-9ec2-ff3d75f025ae-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:29:55.563971 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:55.563941 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" Jul 22 11:29:55.564184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:55.563941 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4tzxk" event={"ID":"f28a6e03-a257-4661-9ec2-ff3d75f025ae","Type":"ContainerDied","Data":"765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab"} Jul 22 11:29:55.564184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:29:55.564062 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="765c3b021dd4f42417ca351499343237ba6796ff9e88ca846f1f444413a03fab" Jul 22 11:30:07.680093 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680050 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f"] Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680461 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="pull" Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680479 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="pull" Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680504 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="extract" Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680512 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="extract" Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680525 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="util" Jul 22 11:30:07.680544 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680531 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="util" Jul 22 11:30:07.680883 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.680634 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f28a6e03-a257-4661-9ec2-ff3d75f025ae" containerName="extract" Jul 22 11:30:07.701766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.701740 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.703118 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.703091 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f"] Jul 22 11:30:07.706365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.706343 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-x6rn5\"" Jul 22 11:30:07.706791 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.706774 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 11:30:07.706885 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.706778 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 11:30:07.707095 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.707079 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 11:30:07.707329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.707315 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 11:30:07.777365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.777323 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-manifests\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.777365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.777359 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-tmp\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.777587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.777382 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.777587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.777406 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.777587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.777455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krztz\" (UniqueName: \"kubernetes.io/projected/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-kube-api-access-krztz\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878372 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-manifests\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878409 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-tmp\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878671 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878433 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878671 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878671 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878640 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krztz\" (UniqueName: \"kubernetes.io/projected/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-kube-api-access-krztz\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.878873 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.878842 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-manifests\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.880919 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.880892 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-tmp\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.881071 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.881050 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-webhook-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.881151 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.881137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:07.890091 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:07.890066 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krztz\" (UniqueName: \"kubernetes.io/projected/965bc4fc-6f0e-4cd9-b506-737b2bca2bcf-kube-api-access-krztz\") pod \"opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f\" (UID: \"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf\") " pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:08.013144 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:08.013055 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:08.196976 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:08.196932 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f"] Jul 22 11:30:08.200395 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:08.199171 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965bc4fc_6f0e_4cd9_b506_737b2bca2bcf.slice/crio-429cd81c3cd73b66a566d2bac03cb9c08da25f043c2f1afb38826e0eb98ee166 WatchSource:0}: Error finding container 429cd81c3cd73b66a566d2bac03cb9c08da25f043c2f1afb38826e0eb98ee166: Status 404 returned error can't find the container with id 429cd81c3cd73b66a566d2bac03cb9c08da25f043c2f1afb38826e0eb98ee166 Jul 22 11:30:08.618162 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:08.618125 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" event={"ID":"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf","Type":"ContainerStarted","Data":"429cd81c3cd73b66a566d2bac03cb9c08da25f043c2f1afb38826e0eb98ee166"} Jul 22 11:30:11.635166 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:11.635130 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" event={"ID":"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf","Type":"ContainerStarted","Data":"e811f0d19667ff78a371d29aab57ea913d6a1460c5d87c853bbebececda3ef56"} Jul 22 11:30:12.640817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:12.640780 2577 generic.go:356] "Generic (PLEG): container finished" podID="965bc4fc-6f0e-4cd9-b506-737b2bca2bcf" containerID="e811f0d19667ff78a371d29aab57ea913d6a1460c5d87c853bbebececda3ef56" exitCode=0 Jul 22 11:30:12.641217 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:12.640847 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" event={"ID":"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf","Type":"ContainerDied","Data":"e811f0d19667ff78a371d29aab57ea913d6a1460c5d87c853bbebececda3ef56"} Jul 22 11:30:13.648089 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:13.648043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" event={"ID":"965bc4fc-6f0e-4cd9-b506-737b2bca2bcf","Type":"ContainerStarted","Data":"f91e96c411d6fe173b1078a2c7365268b70deae83e037e0f0a004b34eafb3ade"} Jul 22 11:30:13.648526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:13.648187 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:13.685779 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:13.685728 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" podStartSLOduration=2.248458307 podStartE2EDuration="6.685711002s" totalImagesPullingTime="3.406004121s" totalInitContainerRuntime="1.031248574s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:13.683720766 +0000 UTC m=+382.191621651" watchObservedRunningTime="2026-07-22 11:30:13.685711002 +0000 UTC m=+382.193611892" Jul 22 11:30:24.656434 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:24.656403 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f" Jul 22 11:30:28.204844 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.204807 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g"] Jul 22 11:30:28.210463 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.210441 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.215002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.214977 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 11:30:28.215644 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.215626 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 11:30:28.215765 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.215627 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-z7lvc\"" Jul 22 11:30:28.221485 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.221459 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g"] Jul 22 11:30:28.355282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.355248 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aef6707e-8546-4916-88fe-129106ba7f03-tmp\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.355461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.355322 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef6707e-8546-4916-88fe-129106ba7f03-tls-certs\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.355461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.355347 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgk7k\" (UniqueName: \"kubernetes.io/projected/aef6707e-8546-4916-88fe-129106ba7f03-kube-api-access-kgk7k\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.456800 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.456704 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef6707e-8546-4916-88fe-129106ba7f03-tls-certs\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.456800 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.456743 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgk7k\" (UniqueName: \"kubernetes.io/projected/aef6707e-8546-4916-88fe-129106ba7f03-kube-api-access-kgk7k\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.456800 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.456786 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aef6707e-8546-4916-88fe-129106ba7f03-tmp\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.459347 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.459324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aef6707e-8546-4916-88fe-129106ba7f03-tmp\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.459461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.459324 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef6707e-8546-4916-88fe-129106ba7f03-tls-certs\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.485097 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.485065 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgk7k\" (UniqueName: \"kubernetes.io/projected/aef6707e-8546-4916-88fe-129106ba7f03-kube-api-access-kgk7k\") pod \"kube-auth-proxy-ffd4cd5bc-wcm8g\" (UID: \"aef6707e-8546-4916-88fe-129106ba7f03\") " pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.521876 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.521842 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" Jul 22 11:30:28.592505 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.591053 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdqv6"] Jul 22 11:30:28.594335 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.594312 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:28.598356 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.598291 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 22 11:30:28.598524 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.598504 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4wv5x\"" Jul 22 11:30:28.610578 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.610550 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdqv6"] Jul 22 11:30:28.659042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.658990 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:28.659042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.659032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7qq5\" (UniqueName: \"kubernetes.io/projected/cf64a448-9766-48b5-b340-c00610a0dcea-kube-api-access-j7qq5\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:28.690214 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.690181 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g"] Jul 22 11:30:28.694364 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:28.694323 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef6707e_8546_4916_88fe_129106ba7f03.slice/crio-59ad77216ed53f9291203da4eec075748cbbeb8e757c06e5844cb5e5c8813fb9 WatchSource:0}: Error finding container 59ad77216ed53f9291203da4eec075748cbbeb8e757c06e5844cb5e5c8813fb9: Status 404 returned error can't find the container with id 59ad77216ed53f9291203da4eec075748cbbeb8e757c06e5844cb5e5c8813fb9 Jul 22 11:30:28.720699 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.720616 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" event={"ID":"aef6707e-8546-4916-88fe-129106ba7f03","Type":"ContainerStarted","Data":"59ad77216ed53f9291203da4eec075748cbbeb8e757c06e5844cb5e5c8813fb9"} Jul 22 11:30:28.760060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.760025 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:28.760060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.760061 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7qq5\" (UniqueName: \"kubernetes.io/projected/cf64a448-9766-48b5-b340-c00610a0dcea-kube-api-access-j7qq5\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:28.760273 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:30:28.760182 2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 22 11:30:28.760273 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:30:28.760257 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert podName:cf64a448-9766-48b5-b340-c00610a0dcea nodeName:}" failed. No retries permitted until 2026-07-22 11:30:29.260239653 +0000 UTC m=+397.768140517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert") pod "odh-model-controller-7998c67dc4-tdqv6" (UID: "cf64a448-9766-48b5-b340-c00610a0dcea") : secret "odh-model-controller-webhook-cert" not found Jul 22 11:30:28.776042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:28.776013 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7qq5\" (UniqueName: \"kubernetes.io/projected/cf64a448-9766-48b5-b340-c00610a0dcea-kube-api-access-j7qq5\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:29.264327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.264287 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:29.267269 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.267244 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf64a448-9766-48b5-b340-c00610a0dcea-cert\") pod \"odh-model-controller-7998c67dc4-tdqv6\" (UID: \"cf64a448-9766-48b5-b340-c00610a0dcea\") " pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:29.505823 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.505786 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:29.730768 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:29.730733 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf64a448_9766_48b5_b340_c00610a0dcea.slice/crio-82edc0f83c77e256da70e16d542af82368b3300d8fdd5e20aac3f35f90924d60 WatchSource:0}: Error finding container 82edc0f83c77e256da70e16d542af82368b3300d8fdd5e20aac3f35f90924d60: Status 404 returned error can't find the container with id 82edc0f83c77e256da70e16d542af82368b3300d8fdd5e20aac3f35f90924d60 Jul 22 11:30:29.752390 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.752362 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tdqv6"] Jul 22 11:30:29.855516 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.855482 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ctx2"] Jul 22 11:30:29.858722 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.858692 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:29.890883 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.890852 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 22 11:30:29.972327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.972262 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdfft\" (UniqueName: \"kubernetes.io/projected/3273357f-2e8d-4a2d-b722-d0161ebe5274-kube-api-access-wdfft\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:29.972520 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:29.972384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.008647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.007817 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ctx2"] Jul 22 11:30:30.073206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.073167 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdfft\" (UniqueName: \"kubernetes.io/projected/3273357f-2e8d-4a2d-b722-d0161ebe5274-kube-api-access-wdfft\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.073389 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.073270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.073455 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:30:30.073429 2577 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 22 11:30:30.073515 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:30:30.073507 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs podName:3273357f-2e8d-4a2d-b722-d0161ebe5274 nodeName:}" failed. No retries permitted until 2026-07-22 11:30:30.57348464 +0000 UTC m=+399.081385507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs") pod "model-serving-api-54d89bd5fc-6ctx2" (UID: "3273357f-2e8d-4a2d-b722-d0161ebe5274") : secret "model-serving-api-tls" not found Jul 22 11:30:30.110710 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.107913 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdfft\" (UniqueName: \"kubernetes.io/projected/3273357f-2e8d-4a2d-b722-d0161ebe5274-kube-api-access-wdfft\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.516458 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.516374 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:30.521553 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.521528 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.525184 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.525159 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 11:30:30.525327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.525188 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 11:30:30.525629 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.525584 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:30:30.525752 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.525662 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 11:30:30.526063 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.526044 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-sfgp6\"" Jul 22 11:30:30.546575 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.546543 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:30.579659 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.579618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.582660 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.582632 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3273357f-2e8d-4a2d-b722-d0161ebe5274-tls-certs\") pod \"model-serving-api-54d89bd5fc-6ctx2\" (UID: \"3273357f-2e8d-4a2d-b722-d0161ebe5274\") " pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.680795 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.680750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.680978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.680807 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.680978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.680880 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd452\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.681098 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.680968 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.681098 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.681046 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.681098 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.681083 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.681261 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.681136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.733287 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.733244 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" event={"ID":"cf64a448-9766-48b5-b340-c00610a0dcea","Type":"ContainerStarted","Data":"82edc0f83c77e256da70e16d542af82368b3300d8fdd5e20aac3f35f90924d60"} Jul 22 11:30:30.773672 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.773565 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:30.782003 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.781972 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782145 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782037 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd452\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782145 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782079 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782145 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782127 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782155 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782184 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.782300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782248 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.783304 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.782992 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.785462 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.785437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.785579 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.785497 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.786708 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.786681 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.787284 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.787246 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.796894 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.796868 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd452\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.798533 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.798505 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token\") pod \"istiod-openshift-gateway-7574459656-n9t8f\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:30.833713 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:30.833675 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:31.204057 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.204029 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:31.207364 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:31.207323 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a09f11d_8dae_4fbd_b0a0_cc3bd690042c.slice/crio-be85394e45c6349bcde13e9661e4562df50df3c7cb96aa331dbf8e96e6d4fbfe WatchSource:0}: Error finding container be85394e45c6349bcde13e9661e4562df50df3c7cb96aa331dbf8e96e6d4fbfe: Status 404 returned error can't find the container with id be85394e45c6349bcde13e9661e4562df50df3c7cb96aa331dbf8e96e6d4fbfe Jul 22 11:30:31.225992 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.225912 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-6ctx2"] Jul 22 11:30:31.229380 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:31.229350 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3273357f_2e8d_4a2d_b722_d0161ebe5274.slice/crio-afea8d5d9ed1cad1c9a09ccaf041358230293fbd616a2f3315ccedc05f29cc76 WatchSource:0}: Error finding container afea8d5d9ed1cad1c9a09ccaf041358230293fbd616a2f3315ccedc05f29cc76: Status 404 returned error can't find the container with id afea8d5d9ed1cad1c9a09ccaf041358230293fbd616a2f3315ccedc05f29cc76 Jul 22 11:30:31.741837 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.741706 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" event={"ID":"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c","Type":"ContainerStarted","Data":"be85394e45c6349bcde13e9661e4562df50df3c7cb96aa331dbf8e96e6d4fbfe"} Jul 22 11:30:31.744362 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.744272 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" event={"ID":"aef6707e-8546-4916-88fe-129106ba7f03","Type":"ContainerStarted","Data":"944030421a0bd1d416aa239fde8fb0e5678c5f66583e17efae9d592a4f32964e"} Jul 22 11:30:31.747212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.747145 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" event={"ID":"3273357f-2e8d-4a2d-b722-d0161ebe5274","Type":"ContainerStarted","Data":"afea8d5d9ed1cad1c9a09ccaf041358230293fbd616a2f3315ccedc05f29cc76"} Jul 22 11:30:31.769767 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:31.768873 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-ffd4cd5bc-wcm8g" podStartSLOduration=1.401516658 podStartE2EDuration="3.768849007s" totalImagesPullingTime="2.367332349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:31.765680023 +0000 UTC m=+400.273580909" watchObservedRunningTime="2026-07-22 11:30:31.768849007 +0000 UTC m=+400.276749894" Jul 22 11:30:35.113213 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.113175 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:30:35.113519 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.113273 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:30:35.769873 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.769832 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" event={"ID":"3273357f-2e8d-4a2d-b722-d0161ebe5274","Type":"ContainerStarted","Data":"e5579095620f195293e88231ade04ae8bc6c8d4c89bbca87a1ef67ab349486a7"} Jul 22 11:30:35.770079 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.769904 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:35.771495 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.771464 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" event={"ID":"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c","Type":"ContainerStarted","Data":"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d"} Jul 22 11:30:35.771614 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.771519 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:35.772881 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.772859 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" event={"ID":"cf64a448-9766-48b5-b340-c00610a0dcea","Type":"ContainerStarted","Data":"5a9cd53844389cc6e93a86157053415e7814fa814c8fc92902b06a25611ca998"} Jul 22 11:30:35.773039 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.773023 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:35.789492 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.789434 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" podStartSLOduration=2.910249564 podStartE2EDuration="6.789414125s" totalImagesPullingTime="3.879164561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:35.78800739 +0000 UTC m=+404.295908276" watchObservedRunningTime="2026-07-22 11:30:35.789414125 +0000 UTC m=+404.297315013" Jul 22 11:30:35.810739 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.810672 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" podStartSLOduration=1.9079210180000001 podStartE2EDuration="5.81065097s" totalImagesPullingTime="3.902729952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:35.809938054 +0000 UTC m=+404.317838947" watchObservedRunningTime="2026-07-22 11:30:35.81065097 +0000 UTC m=+404.318551857" Jul 22 11:30:35.829220 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:35.829163 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" podStartSLOduration=2.439503358 podStartE2EDuration="7.829148135s" totalImagesPullingTime="5.389644777s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:35.827351124 +0000 UTC m=+404.335252010" watchObservedRunningTime="2026-07-22 11:30:35.829148135 +0000 UTC m=+404.337049021" Jul 22 11:30:36.780391 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:36.780343 2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-n9t8f container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 11:30:36.780866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:36.780415 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 11:30:38.835683 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.835643 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb"] Jul 22 11:30:38.838936 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.838915 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:38.841695 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841671 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mbp5b\"" Jul 22 11:30:38.841819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841725 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 11:30:38.841819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841759 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 11:30:38.841918 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841899 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 11:30:38.841918 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841912 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 11:30:38.841987 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.841974 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 11:30:38.848709 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.848687 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb"] Jul 22 11:30:38.964991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.964955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:38.965171 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.965005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f5b5be6f-7aa1-4b78-a68e-77894363fba0-manager-config\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:38.965171 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.965145 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrmj7\" (UniqueName: \"kubernetes.io/projected/f5b5be6f-7aa1-4b78-a68e-77894363fba0-kube-api-access-rrmj7\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:38.965320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:38.965217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-metrics-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.066149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.066118 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrmj7\" (UniqueName: \"kubernetes.io/projected/f5b5be6f-7aa1-4b78-a68e-77894363fba0-kube-api-access-rrmj7\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.066329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.066169 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-metrics-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.066329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.066217 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.066329 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.066253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f5b5be6f-7aa1-4b78-a68e-77894363fba0-manager-config\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.066982 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.066944 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f5b5be6f-7aa1-4b78-a68e-77894363fba0-manager-config\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.069362 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.069339 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.069535 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.069514 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f5b5be6f-7aa1-4b78-a68e-77894363fba0-metrics-cert\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.085365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.085335 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrmj7\" (UniqueName: \"kubernetes.io/projected/f5b5be6f-7aa1-4b78-a68e-77894363fba0-kube-api-access-rrmj7\") pod \"lws-controller-manager-55d777d8cf-xlzqb\" (UID: \"f5b5be6f-7aa1-4b78-a68e-77894363fba0\") " pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.148638 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.148587 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:39.308060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.308031 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb"] Jul 22 11:30:39.309861 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:30:39.309823 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b5be6f_7aa1_4b78_a68e_77894363fba0.slice/crio-729f7cb4d1650463d7dc55a6ba6a57d20f1a80b0b1b856d7fa08aeddf0ce5d3f WatchSource:0}: Error finding container 729f7cb4d1650463d7dc55a6ba6a57d20f1a80b0b1b856d7fa08aeddf0ce5d3f: Status 404 returned error can't find the container with id 729f7cb4d1650463d7dc55a6ba6a57d20f1a80b0b1b856d7fa08aeddf0ce5d3f Jul 22 11:30:39.779841 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.779813 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:39.794244 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:39.794220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" event={"ID":"f5b5be6f-7aa1-4b78-a68e-77894363fba0","Type":"ContainerStarted","Data":"729f7cb4d1650463d7dc55a6ba6a57d20f1a80b0b1b856d7fa08aeddf0ce5d3f"} Jul 22 11:30:41.808520 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:41.808486 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" event={"ID":"f5b5be6f-7aa1-4b78-a68e-77894363fba0","Type":"ContainerStarted","Data":"41e0e6bc726416e4125b9655d8a9c7cdec90c8c59645e6ea4f8f9d301fd51a08"} Jul 22 11:30:41.808972 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:41.808639 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:30:41.853620 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:41.853547 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" podStartSLOduration=2.275317436 podStartE2EDuration="3.853530214s" totalImagesPullingTime="1.578212778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:30:41.852427972 +0000 UTC m=+410.360328859" watchObservedRunningTime="2026-07-22 11:30:41.853530214 +0000 UTC m=+410.361431097" Jul 22 11:30:43.207502 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.207469 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:43.207935 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.207704 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerName="discovery" containerID="cri-o://95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d" gracePeriod=30 Jul 22 11:30:43.461451 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.461391 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:43.609210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609172 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609208 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609236 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609256 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609300 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609331 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609382 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd452\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452\") pod \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\" (UID: \"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c\") " Jul 22 11:30:43.609840 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.609805 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:30:43.611994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.611965 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:30:43.612111 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.611989 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452" (OuterVolumeSpecName: "kube-api-access-gd452") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "kube-api-access-gd452". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:30:43.612190 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.612171 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:30:43.612264 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.612246 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs" (OuterVolumeSpecName: "local-certs") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:30:43.612401 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.612381 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:30:43.612488 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.612403 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts" (OuterVolumeSpecName: "cacerts") pod "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" (UID: "2a09f11d-8dae-4fbd-b0a0-cc3bd690042c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:30:43.711060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711022 2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-local-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711051 2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-kubeconfig\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711060 2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-cacerts\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711068 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711076 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-ca-configmap\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711087 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-istio-csr-dns-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.711312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.711096 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd452\" (UniqueName: \"kubernetes.io/projected/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c-kube-api-access-gd452\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:30:43.825756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.821224 2577 generic.go:356] "Generic (PLEG): container finished" podID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerID="95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d" exitCode=0 Jul 22 11:30:43.825756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.821328 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" event={"ID":"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c","Type":"ContainerDied","Data":"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d"} Jul 22 11:30:43.825756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.821373 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" event={"ID":"2a09f11d-8dae-4fbd-b0a0-cc3bd690042c","Type":"ContainerDied","Data":"be85394e45c6349bcde13e9661e4562df50df3c7cb96aa331dbf8e96e6d4fbfe"} Jul 22 11:30:43.825756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.821398 2577 scope.go:117] "RemoveContainer" containerID="95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d" Jul 22 11:30:43.825756 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.821558 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f" Jul 22 11:30:43.836691 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.836657 2577 scope.go:117] "RemoveContainer" containerID="95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d" Jul 22 11:30:43.836956 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:30:43.836934 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d\": container with ID starting with 95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d not found: ID does not exist" containerID="95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d" Jul 22 11:30:43.837021 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.836965 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d"} err="failed to get container status \"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d\": rpc error: code = NotFound desc = could not find container \"95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d\": container with ID starting with 95af3a8793663de27422702db862eade03271fd21e41be9bcfe133ca8826929d not found: ID does not exist" Jul 22 11:30:43.852746 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.852713 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:43.855980 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:43.855955 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-n9t8f"] Jul 22 11:30:44.066919 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:44.066888 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" path="/var/lib/kubelet/pods/2a09f11d-8dae-4fbd-b0a0-cc3bd690042c/volumes" Jul 22 11:30:46.780421 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:46.780392 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tdqv6" Jul 22 11:30:46.783362 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:46.783340 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-6ctx2" Jul 22 11:30:52.815944 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:30:52.815908 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55d777d8cf-xlzqb" Jul 22 11:31:44.177701 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.177662 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb"] Jul 22 11:31:44.178109 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.177959 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerName="discovery" Jul 22 11:31:44.178109 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.177970 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerName="discovery" Jul 22 11:31:44.178109 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.178036 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a09f11d-8dae-4fbd-b0a0-cc3bd690042c" containerName="discovery" Jul 22 11:31:44.179851 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.179835 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.182273 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.182245 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 11:31:44.182273 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.182263 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 11:31:44.182485 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.182410 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hc5sn\"" Jul 22 11:31:44.189681 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.189660 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb"] Jul 22 11:31:44.262428 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.262395 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.262617 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.262438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hs9t\" (UniqueName: \"kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.262617 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.262459 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.363908 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.363870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.364064 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.363922 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hs9t\" (UniqueName: \"kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.364064 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.363957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.364275 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.364256 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.364310 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.364289 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.373817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.373794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hs9t\" (UniqueName: \"kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.489638 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.489530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:44.579629 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.579582 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs"] Jul 22 11:31:44.582891 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.582868 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.591539 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.591430 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs"] Jul 22 11:31:44.632386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.632364 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb"] Jul 22 11:31:44.634833 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:31:44.634808 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589004d7_85c3_42ad_b4a9_eab9c7cff7cc.slice/crio-57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d WatchSource:0}: Error finding container 57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d: Status 404 returned error can't find the container with id 57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d Jul 22 11:31:44.667179 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.667144 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zssw4\" (UniqueName: \"kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.667289 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.667187 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.667289 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.667227 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.768567 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.768532 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zssw4\" (UniqueName: \"kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.768718 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.768582 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.768718 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.768664 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.768980 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.768963 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.769023 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.768990 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.778842 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.778817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zssw4\" (UniqueName: \"kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.895975 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.895944 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:44.977556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.977523 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4"] Jul 22 11:31:44.980991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.980938 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:44.992015 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:44.991480 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4"] Jul 22 11:31:45.042267 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.042235 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs"] Jul 22 11:31:45.044221 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:31:45.044196 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e62feb_ff53_4e0e_9344_42105dde1e37.slice/crio-ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c WatchSource:0}: Error finding container ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c: Status 404 returned error can't find the container with id ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c Jul 22 11:31:45.071369 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.071342 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.071472 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.071381 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5gw2\" (UniqueName: \"kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.071533 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.071474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.099898 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.099824 2577 generic.go:356] "Generic (PLEG): container finished" podID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerID="3844943492a244792f0f84bbb473e0bf1e2f360239efb1f69fdcc55402ad9f2d" exitCode=0 Jul 22 11:31:45.099898 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.099902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" event={"ID":"589004d7-85c3-42ad-b4a9-eab9c7cff7cc","Type":"ContainerDied","Data":"3844943492a244792f0f84bbb473e0bf1e2f360239efb1f69fdcc55402ad9f2d"} Jul 22 11:31:45.099898 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.099946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" event={"ID":"589004d7-85c3-42ad-b4a9-eab9c7cff7cc","Type":"ContainerStarted","Data":"57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d"} Jul 22 11:31:45.101649 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.101628 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" event={"ID":"e7e62feb-ff53-4e0e-9344-42105dde1e37","Type":"ContainerStarted","Data":"ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c"} Jul 22 11:31:45.172925 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.172899 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.173053 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.172974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.173053 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.172991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5gw2\" (UniqueName: \"kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.173274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.173245 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.173379 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.173360 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.183410 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.183392 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5gw2\" (UniqueName: \"kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.293939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.293862 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:45.414991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.414953 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7"] Jul 22 11:31:45.418486 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.418465 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.432461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.432431 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7"] Jul 22 11:31:45.463692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.463662 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4"] Jul 22 11:31:45.465983 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:31:45.465958 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e34d5f6_bc93_4215_a098_d82c3331cd67.slice/crio-7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e WatchSource:0}: Error finding container 7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e: Status 404 returned error can't find the container with id 7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e Jul 22 11:31:45.474979 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.474955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.475080 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.475031 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.475080 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.475062 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv9cn\" (UniqueName: \"kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.576039 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.576008 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.576191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.576053 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv9cn\" (UniqueName: \"kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.576191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.576121 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.576491 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.576380 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.576555 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.576513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.588966 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.588943 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv9cn\" (UniqueName: \"kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.727282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.727248 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:45.885907 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:45.885882 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7"] Jul 22 11:31:45.919402 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:31:45.919370 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb273fe00_7a8d_4b05_9a17_d13a51adf853.slice/crio-974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61 WatchSource:0}: Error finding container 974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61: Status 404 returned error can't find the container with id 974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61 Jul 22 11:31:46.107693 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.107660 2577 generic.go:356] "Generic (PLEG): container finished" podID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerID="061383c6b1dc16dd4919c550206daef035f5874ab0f7120c1133e48c14a195e3" exitCode=0 Jul 22 11:31:46.107848 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.107749 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" event={"ID":"b273fe00-7a8d-4b05-9a17-d13a51adf853","Type":"ContainerDied","Data":"061383c6b1dc16dd4919c550206daef035f5874ab0f7120c1133e48c14a195e3"} Jul 22 11:31:46.107848 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.107802 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" event={"ID":"b273fe00-7a8d-4b05-9a17-d13a51adf853","Type":"ContainerStarted","Data":"974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61"} Jul 22 11:31:46.109580 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.109557 2577 generic.go:356] "Generic (PLEG): container finished" podID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerID="5db8533909bd2cd634399e48cd8e801ae651d7d3966c601fb239b0455587df8c" exitCode=0 Jul 22 11:31:46.109701 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.109628 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" event={"ID":"589004d7-85c3-42ad-b4a9-eab9c7cff7cc","Type":"ContainerDied","Data":"5db8533909bd2cd634399e48cd8e801ae651d7d3966c601fb239b0455587df8c"} Jul 22 11:31:46.111268 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.111245 2577 generic.go:356] "Generic (PLEG): container finished" podID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerID="4615c514cc5165edcf6b28cef799017821c0ae05f7f79c2c3735b37e8139c2dc" exitCode=0 Jul 22 11:31:46.111366 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.111337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" event={"ID":"1e34d5f6-bc93-4215-a098-d82c3331cd67","Type":"ContainerDied","Data":"4615c514cc5165edcf6b28cef799017821c0ae05f7f79c2c3735b37e8139c2dc"} Jul 22 11:31:46.111432 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.111377 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" event={"ID":"1e34d5f6-bc93-4215-a098-d82c3331cd67","Type":"ContainerStarted","Data":"7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e"} Jul 22 11:31:46.112888 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.112869 2577 generic.go:356] "Generic (PLEG): container finished" podID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerID="d0552603532aeaddee02959c3bb4eba5cfbe2b80d654098f06f381ca676d48de" exitCode=0 Jul 22 11:31:46.113001 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:46.112930 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" event={"ID":"e7e62feb-ff53-4e0e-9344-42105dde1e37","Type":"ContainerDied","Data":"d0552603532aeaddee02959c3bb4eba5cfbe2b80d654098f06f381ca676d48de"} Jul 22 11:31:47.119133 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.119102 2577 generic.go:356] "Generic (PLEG): container finished" podID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerID="154b8b58f11ff980ef15583a01852c0a12d27d8f7617dd91e6c9e4763439b85e" exitCode=0 Jul 22 11:31:47.119494 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.119186 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" event={"ID":"b273fe00-7a8d-4b05-9a17-d13a51adf853","Type":"ContainerDied","Data":"154b8b58f11ff980ef15583a01852c0a12d27d8f7617dd91e6c9e4763439b85e"} Jul 22 11:31:47.121354 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.121332 2577 generic.go:356] "Generic (PLEG): container finished" podID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerID="f0a2ba7052933d1589bd35b7c2ff86328296496e6215027fb6a7162303e330d7" exitCode=0 Jul 22 11:31:47.121457 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.121392 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" event={"ID":"589004d7-85c3-42ad-b4a9-eab9c7cff7cc","Type":"ContainerDied","Data":"f0a2ba7052933d1589bd35b7c2ff86328296496e6215027fb6a7162303e330d7"} Jul 22 11:31:47.123047 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.123028 2577 generic.go:356] "Generic (PLEG): container finished" podID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerID="84c1bcf4328f57988d9c44c85dad53a8c2a7fde784a9e74ffa8f5f41f5bcae60" exitCode=0 Jul 22 11:31:47.123150 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.123118 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" event={"ID":"1e34d5f6-bc93-4215-a098-d82c3331cd67","Type":"ContainerDied","Data":"84c1bcf4328f57988d9c44c85dad53a8c2a7fde784a9e74ffa8f5f41f5bcae60"} Jul 22 11:31:47.124900 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.124880 2577 generic.go:356] "Generic (PLEG): container finished" podID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerID="844109e8a7f4ef793f88f99db822c76ae68694739668293990199f6623ea0632" exitCode=0 Jul 22 11:31:47.124993 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:47.124937 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" event={"ID":"e7e62feb-ff53-4e0e-9344-42105dde1e37","Type":"ContainerDied","Data":"844109e8a7f4ef793f88f99db822c76ae68694739668293990199f6623ea0632"} Jul 22 11:31:48.131319 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.131286 2577 generic.go:356] "Generic (PLEG): container finished" podID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerID="5ac7d0cb36013224e3435618ba65d8e34ad2df4327a297e2362c911aab22c534" exitCode=0 Jul 22 11:31:48.131748 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.131388 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" event={"ID":"b273fe00-7a8d-4b05-9a17-d13a51adf853","Type":"ContainerDied","Data":"5ac7d0cb36013224e3435618ba65d8e34ad2df4327a297e2362c911aab22c534"} Jul 22 11:31:48.133222 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.133203 2577 generic.go:356] "Generic (PLEG): container finished" podID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerID="11af8867debf4425e5a87fceffb5607185872f87b2367e7e65047b33faff38b9" exitCode=0 Jul 22 11:31:48.133310 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.133287 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" event={"ID":"1e34d5f6-bc93-4215-a098-d82c3331cd67","Type":"ContainerDied","Data":"11af8867debf4425e5a87fceffb5607185872f87b2367e7e65047b33faff38b9"} Jul 22 11:31:48.135162 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.135140 2577 generic.go:356] "Generic (PLEG): container finished" podID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerID="2eb867251def75b2538b5681749b82d79378b2d6686c3e5b709fe6288bcacbf7" exitCode=0 Jul 22 11:31:48.135278 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.135163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" event={"ID":"e7e62feb-ff53-4e0e-9344-42105dde1e37","Type":"ContainerDied","Data":"2eb867251def75b2538b5681749b82d79378b2d6686c3e5b709fe6288bcacbf7"} Jul 22 11:31:48.268208 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.268175 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:48.400717 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.400627 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle\") pod \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " Jul 22 11:31:48.400717 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.400671 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util\") pod \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " Jul 22 11:31:48.400717 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.400715 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hs9t\" (UniqueName: \"kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t\") pod \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\" (UID: \"589004d7-85c3-42ad-b4a9-eab9c7cff7cc\") " Jul 22 11:31:48.401209 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.401175 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle" (OuterVolumeSpecName: "bundle") pod "589004d7-85c3-42ad-b4a9-eab9c7cff7cc" (UID: "589004d7-85c3-42ad-b4a9-eab9c7cff7cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:48.403122 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.403099 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t" (OuterVolumeSpecName: "kube-api-access-7hs9t") pod "589004d7-85c3-42ad-b4a9-eab9c7cff7cc" (UID: "589004d7-85c3-42ad-b4a9-eab9c7cff7cc"). InnerVolumeSpecName "kube-api-access-7hs9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:31:48.409736 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.409707 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util" (OuterVolumeSpecName: "util") pod "589004d7-85c3-42ad-b4a9-eab9c7cff7cc" (UID: "589004d7-85c3-42ad-b4a9-eab9c7cff7cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:48.501352 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.501316 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:48.501352 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.501345 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:48.501352 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:48.501355 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hs9t\" (UniqueName: \"kubernetes.io/projected/589004d7-85c3-42ad-b4a9-eab9c7cff7cc-kube-api-access-7hs9t\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.144345 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.144315 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" Jul 22 11:31:49.144819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.144367 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb" event={"ID":"589004d7-85c3-42ad-b4a9-eab9c7cff7cc","Type":"ContainerDied","Data":"57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d"} Jul 22 11:31:49.144819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.144416 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="57b529b650b55ece1f4259062f022c6a16382a61c40d8c60787afffe27509b5d" Jul 22 11:31:49.289073 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.289046 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:49.330842 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.330817 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:49.334017 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.333998 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:49.409218 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409116 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5gw2\" (UniqueName: \"kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2\") pod \"1e34d5f6-bc93-4215-a098-d82c3331cd67\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " Jul 22 11:31:49.409218 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409174 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util\") pod \"b273fe00-7a8d-4b05-9a17-d13a51adf853\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " Jul 22 11:31:49.409218 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409211 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle\") pod \"1e34d5f6-bc93-4215-a098-d82c3331cd67\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409251 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv9cn\" (UniqueName: \"kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn\") pod \"b273fe00-7a8d-4b05-9a17-d13a51adf853\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409286 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle\") pod \"e7e62feb-ff53-4e0e-9344-42105dde1e37\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409319 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util\") pod \"e7e62feb-ff53-4e0e-9344-42105dde1e37\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409353 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle\") pod \"b273fe00-7a8d-4b05-9a17-d13a51adf853\" (UID: \"b273fe00-7a8d-4b05-9a17-d13a51adf853\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409389 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zssw4\" (UniqueName: \"kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4\") pod \"e7e62feb-ff53-4e0e-9344-42105dde1e37\" (UID: \"e7e62feb-ff53-4e0e-9344-42105dde1e37\") " Jul 22 11:31:49.409484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409458 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util\") pod \"1e34d5f6-bc93-4215-a098-d82c3331cd67\" (UID: \"1e34d5f6-bc93-4215-a098-d82c3331cd67\") " Jul 22 11:31:49.410356 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.409818 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle" (OuterVolumeSpecName: "bundle") pod "e7e62feb-ff53-4e0e-9344-42105dde1e37" (UID: "e7e62feb-ff53-4e0e-9344-42105dde1e37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.411771 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.410706 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle" (OuterVolumeSpecName: "bundle") pod "b273fe00-7a8d-4b05-9a17-d13a51adf853" (UID: "b273fe00-7a8d-4b05-9a17-d13a51adf853"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.411771 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.411207 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle" (OuterVolumeSpecName: "bundle") pod "1e34d5f6-bc93-4215-a098-d82c3331cd67" (UID: "1e34d5f6-bc93-4215-a098-d82c3331cd67"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.413213 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.413174 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4" (OuterVolumeSpecName: "kube-api-access-zssw4") pod "e7e62feb-ff53-4e0e-9344-42105dde1e37" (UID: "e7e62feb-ff53-4e0e-9344-42105dde1e37"). InnerVolumeSpecName "kube-api-access-zssw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:31:49.413932 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.413897 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2" (OuterVolumeSpecName: "kube-api-access-g5gw2") pod "1e34d5f6-bc93-4215-a098-d82c3331cd67" (UID: "1e34d5f6-bc93-4215-a098-d82c3331cd67"). InnerVolumeSpecName "kube-api-access-g5gw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:31:49.413932 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.413922 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn" (OuterVolumeSpecName: "kube-api-access-hv9cn") pod "b273fe00-7a8d-4b05-9a17-d13a51adf853" (UID: "b273fe00-7a8d-4b05-9a17-d13a51adf853"). InnerVolumeSpecName "kube-api-access-hv9cn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:31:49.418330 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.418303 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util" (OuterVolumeSpecName: "util") pod "1e34d5f6-bc93-4215-a098-d82c3331cd67" (UID: "1e34d5f6-bc93-4215-a098-d82c3331cd67"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.418568 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.418550 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util" (OuterVolumeSpecName: "util") pod "b273fe00-7a8d-4b05-9a17-d13a51adf853" (UID: "b273fe00-7a8d-4b05-9a17-d13a51adf853"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.419021 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.419003 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util" (OuterVolumeSpecName: "util") pod "e7e62feb-ff53-4e0e-9344-42105dde1e37" (UID: "e7e62feb-ff53-4e0e-9344-42105dde1e37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:31:49.510984 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.510944 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5gw2\" (UniqueName: \"kubernetes.io/projected/1e34d5f6-bc93-4215-a098-d82c3331cd67-kube-api-access-g5gw2\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.510984 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.510977 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.510984 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.510987 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.510995 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv9cn\" (UniqueName: \"kubernetes.io/projected/b273fe00-7a8d-4b05-9a17-d13a51adf853-kube-api-access-hv9cn\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.511005 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.511013 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e7e62feb-ff53-4e0e-9344-42105dde1e37-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.511020 2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b273fe00-7a8d-4b05-9a17-d13a51adf853-bundle\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.511027 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zssw4\" (UniqueName: \"kubernetes.io/projected/e7e62feb-ff53-4e0e-9344-42105dde1e37-kube-api-access-zssw4\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:49.511216 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:49.511034 2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e34d5f6-bc93-4215-a098-d82c3331cd67-util\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:31:50.151510 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.151480 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" Jul 22 11:31:50.151947 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.151475 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7" event={"ID":"b273fe00-7a8d-4b05-9a17-d13a51adf853","Type":"ContainerDied","Data":"974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61"} Jul 22 11:31:50.151947 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.151624 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="974f78c8f5a3e8864549894db2143b08f457efd712b39ff27cc440a915ce4a61" Jul 22 11:31:50.153416 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.153394 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" Jul 22 11:31:50.153652 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.153621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4" event={"ID":"1e34d5f6-bc93-4215-a098-d82c3331cd67","Type":"ContainerDied","Data":"7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e"} Jul 22 11:31:50.153762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.153689 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7bdaa65a593831846b2d51f4cc682e7db928a5987a17bd7141f8445eb1e83f5e" Jul 22 11:31:50.155489 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.155463 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" event={"ID":"e7e62feb-ff53-4e0e-9344-42105dde1e37","Type":"ContainerDied","Data":"ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c"} Jul 22 11:31:50.155633 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.155499 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs" Jul 22 11:31:50.155633 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:31:50.155505 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed54b99144d80247da0844280aa39a139a85fb96dffa43ea35fa93f4c967f25c" Jul 22 11:32:17.146664 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.146569 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg"] Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147054 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="extract" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147069 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="extract" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147081 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="pull" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147089 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="pull" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147098 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="extract" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147107 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="extract" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147117 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="pull" Jul 22 11:32:17.147129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147125 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="pull" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147137 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="pull" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147144 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="pull" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147158 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147165 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147179 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147187 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147194 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147202 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147214 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147223 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147247 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147254 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="util" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147263 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147271 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147281 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="pull" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147288 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="pull" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147353 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="589004d7-85c3-42ad-b4a9-eab9c7cff7cc" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147362 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7e62feb-ff53-4e0e-9344-42105dde1e37" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147372 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e34d5f6-bc93-4215-a098-d82c3331cd67" containerName="extract" Jul 22 11:32:17.147386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.147381 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b273fe00-7a8d-4b05-9a17-d13a51adf853" containerName="extract" Jul 22 11:32:17.150693 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.150675 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.154321 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.154296 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 11:32:17.154776 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.154758 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 11:32:17.154907 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.154794 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 11:32:17.156004 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.155986 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hc5sn\"" Jul 22 11:32:17.156835 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.156816 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 11:32:17.166865 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.166844 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg"] Jul 22 11:32:17.248425 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.248387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8m7t\" (UniqueName: \"kubernetes.io/projected/b5898502-f9b7-45e2-af5e-7b04ec574a9d-kube-api-access-d8m7t\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.248607 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.248457 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.248607 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.248494 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b5898502-f9b7-45e2-af5e-7b04ec574a9d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.349689 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.349655 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8m7t\" (UniqueName: \"kubernetes.io/projected/b5898502-f9b7-45e2-af5e-7b04ec574a9d-kube-api-access-d8m7t\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.349866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.349709 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.349917 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:32:17.349892 2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 22 11:32:17.350343 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:32:17.350321 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert podName:b5898502-f9b7-45e2-af5e-7b04ec574a9d nodeName:}" failed. No retries permitted until 2026-07-22 11:32:17.850292261 +0000 UTC m=+506.358193127 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-2hhsg" (UID: "b5898502-f9b7-45e2-af5e-7b04ec574a9d") : secret "plugin-serving-cert" not found Jul 22 11:32:17.356037 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.355989 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b5898502-f9b7-45e2-af5e-7b04ec574a9d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.357051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.357027 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b5898502-f9b7-45e2-af5e-7b04ec574a9d-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.361804 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.361785 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8m7t\" (UniqueName: \"kubernetes.io/projected/b5898502-f9b7-45e2-af5e-7b04ec574a9d-kube-api-access-d8m7t\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.860022 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.859983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:17.862515 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:17.862485 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5898502-f9b7-45e2-af5e-7b04ec574a9d-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2hhsg\" (UID: \"b5898502-f9b7-45e2-af5e-7b04ec574a9d\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:18.060228 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:18.060188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" Jul 22 11:32:18.198947 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:18.198921 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg"] Jul 22 11:32:18.200699 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:32:18.200672 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5898502_f9b7_45e2_af5e_7b04ec574a9d.slice/crio-fd1bd3e64f83259c3fe0637282e02441be0f628c01d970bad6c08d53e47909f7 WatchSource:0}: Error finding container fd1bd3e64f83259c3fe0637282e02441be0f628c01d970bad6c08d53e47909f7: Status 404 returned error can't find the container with id fd1bd3e64f83259c3fe0637282e02441be0f628c01d970bad6c08d53e47909f7 Jul 22 11:32:18.294540 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:18.294506 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" event={"ID":"b5898502-f9b7-45e2-af5e-7b04ec574a9d","Type":"ContainerStarted","Data":"fd1bd3e64f83259c3fe0637282e02441be0f628c01d970bad6c08d53e47909f7"} Jul 22 11:32:42.425125 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:42.425028 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" event={"ID":"b5898502-f9b7-45e2-af5e-7b04ec574a9d","Type":"ContainerStarted","Data":"833614807a391a04307bc028e36b9391570b0720927c54b7bb73d50811b81a40"} Jul 22 11:32:42.446034 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:32:42.445983 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2hhsg" podStartSLOduration=2.196783004 podStartE2EDuration="25.445967762s" totalImagesPullingTime="23.249184758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:32:42.44410032 +0000 UTC m=+530.952001206" watchObservedRunningTime="2026-07-22 11:32:42.445967762 +0000 UTC m=+530.953868847" Jul 22 11:33:15.595872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.595836 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:15.598082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.598066 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:15.600155 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.600130 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cplr5\"" Jul 22 11:33:15.606905 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.606878 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:15.678714 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.678680 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mscc\" (UniqueName: \"kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc\") pod \"authorino-58cd955675-g7b47\" (UID: \"392f8c50-4765-40d1-af20-096a00691212\") " pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:15.780123 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.780065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mscc\" (UniqueName: \"kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc\") pod \"authorino-58cd955675-g7b47\" (UID: \"392f8c50-4765-40d1-af20-096a00691212\") " pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:15.790735 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.790711 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mscc\" (UniqueName: \"kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc\") pod \"authorino-58cd955675-g7b47\" (UID: \"392f8c50-4765-40d1-af20-096a00691212\") " pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:15.804286 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.804262 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:15.806462 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.806445 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:15.816939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.816914 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:15.881537 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.881500 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgj9r\" (UniqueName: \"kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r\") pod \"authorino-75d556bd77-8ltrb\" (UID: \"c001157a-05d7-4f57-bbd4-12fc3a551afa\") " pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:15.908365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.908336 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:15.982843 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.982806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgj9r\" (UniqueName: \"kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r\") pod \"authorino-75d556bd77-8ltrb\" (UID: \"c001157a-05d7-4f57-bbd4-12fc3a551afa\") " pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:15.993246 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:15.993205 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgj9r\" (UniqueName: \"kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r\") pod \"authorino-75d556bd77-8ltrb\" (UID: \"c001157a-05d7-4f57-bbd4-12fc3a551afa\") " pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:16.046649 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:16.046621 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:16.048829 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:33:16.048802 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod392f8c50_4765_40d1_af20_096a00691212.slice/crio-a476c3d87ebe49f8fe24c32101967c6490e63038165f6d396ff8d4addd6fc2c4 WatchSource:0}: Error finding container a476c3d87ebe49f8fe24c32101967c6490e63038165f6d396ff8d4addd6fc2c4: Status 404 returned error can't find the container with id a476c3d87ebe49f8fe24c32101967c6490e63038165f6d396ff8d4addd6fc2c4 Jul 22 11:33:16.115881 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:16.115853 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:16.254435 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:16.254412 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:16.256205 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:33:16.256178 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc001157a_05d7_4f57_bbd4_12fc3a551afa.slice/crio-65126e287d8a9d5b1adbbcaf746ba602c0d431923f304d1c48922f7468bc9a8e WatchSource:0}: Error finding container 65126e287d8a9d5b1adbbcaf746ba602c0d431923f304d1c48922f7468bc9a8e: Status 404 returned error can't find the container with id 65126e287d8a9d5b1adbbcaf746ba602c0d431923f304d1c48922f7468bc9a8e Jul 22 11:33:16.591235 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:16.591134 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g7b47" event={"ID":"392f8c50-4765-40d1-af20-096a00691212","Type":"ContainerStarted","Data":"a476c3d87ebe49f8fe24c32101967c6490e63038165f6d396ff8d4addd6fc2c4"} Jul 22 11:33:16.592282 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:16.592254 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8ltrb" event={"ID":"c001157a-05d7-4f57-bbd4-12fc3a551afa","Type":"ContainerStarted","Data":"65126e287d8a9d5b1adbbcaf746ba602c0d431923f304d1c48922f7468bc9a8e"} Jul 22 11:33:20.619112 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:20.619074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g7b47" event={"ID":"392f8c50-4765-40d1-af20-096a00691212","Type":"ContainerStarted","Data":"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3"} Jul 22 11:33:20.620530 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:20.620508 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8ltrb" event={"ID":"c001157a-05d7-4f57-bbd4-12fc3a551afa","Type":"ContainerStarted","Data":"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8"} Jul 22 11:33:20.635861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:20.635812 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-g7b47" podStartSLOduration=1.957689807 podStartE2EDuration="5.635797804s" totalImagesPullingTime="3.678107997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:33:20.635161288 +0000 UTC m=+569.143062174" watchObservedRunningTime="2026-07-22 11:33:20.635797804 +0000 UTC m=+569.143698690" Jul 22 11:33:20.650701 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:20.650641 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-8ltrb" podStartSLOduration=2.169552043 podStartE2EDuration="5.650620538s" totalImagesPullingTime="3.481068495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:33:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:33:20.650370532 +0000 UTC m=+569.158271414" watchObservedRunningTime="2026-07-22 11:33:20.650620538 +0000 UTC m=+569.158521424" Jul 22 11:33:20.681001 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:20.680967 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:22.631365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:22.631318 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-g7b47" podUID="392f8c50-4765-40d1-af20-096a00691212" containerName="authorino" containerID="cri-o://0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3" gracePeriod=30 Jul 22 11:33:22.881895 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:22.881837 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:22.949833 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:22.949805 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mscc\" (UniqueName: \"kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc\") pod \"392f8c50-4765-40d1-af20-096a00691212\" (UID: \"392f8c50-4765-40d1-af20-096a00691212\") " Jul 22 11:33:22.951933 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:22.951899 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc" (OuterVolumeSpecName: "kube-api-access-8mscc") pod "392f8c50-4765-40d1-af20-096a00691212" (UID: "392f8c50-4765-40d1-af20-096a00691212"). InnerVolumeSpecName "kube-api-access-8mscc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:33:23.050482 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.050447 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mscc\" (UniqueName: \"kubernetes.io/projected/392f8c50-4765-40d1-af20-096a00691212-kube-api-access-8mscc\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:33:23.637447 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.637411 2577 generic.go:356] "Generic (PLEG): container finished" podID="392f8c50-4765-40d1-af20-096a00691212" containerID="0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3" exitCode=0 Jul 22 11:33:23.637447 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.637450 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-g7b47" Jul 22 11:33:23.637912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.637490 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g7b47" event={"ID":"392f8c50-4765-40d1-af20-096a00691212","Type":"ContainerDied","Data":"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3"} Jul 22 11:33:23.637912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.637524 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-g7b47" event={"ID":"392f8c50-4765-40d1-af20-096a00691212","Type":"ContainerDied","Data":"a476c3d87ebe49f8fe24c32101967c6490e63038165f6d396ff8d4addd6fc2c4"} Jul 22 11:33:23.637912 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.637544 2577 scope.go:117] "RemoveContainer" containerID="0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3" Jul 22 11:33:23.646607 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.646578 2577 scope.go:117] "RemoveContainer" containerID="0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3" Jul 22 11:33:23.646866 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:33:23.646848 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3\": container with ID starting with 0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3 not found: ID does not exist" containerID="0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3" Jul 22 11:33:23.646925 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.646874 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3"} err="failed to get container status \"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3\": rpc error: code = NotFound desc = could not find container \"0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3\": container with ID starting with 0d924d401f3cc9950ff678b1f80b5277fc7befea7869f8be62a0025be1c81fd3 not found: ID does not exist" Jul 22 11:33:23.659124 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.659103 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:23.663359 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:23.663340 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-g7b47"] Jul 22 11:33:24.066927 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:24.066848 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="392f8c50-4765-40d1-af20-096a00691212" path="/var/lib/kubelet/pods/392f8c50-4765-40d1-af20-096a00691212/volumes" Jul 22 11:33:47.571566 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.571527 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:47.572061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.571885 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="392f8c50-4765-40d1-af20-096a00691212" containerName="authorino" Jul 22 11:33:47.572061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.571897 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="392f8c50-4765-40d1-af20-096a00691212" containerName="authorino" Jul 22 11:33:47.572061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.571949 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="392f8c50-4765-40d1-af20-096a00691212" containerName="authorino" Jul 22 11:33:47.573778 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.573760 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:47.585862 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.585838 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:47.655361 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.655320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lss5w\" (UniqueName: \"kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w\") pod \"authorino-5d4b67c446-cxxxw\" (UID: \"5c3e80c7-97db-4e6b-a9d5-377a806e9978\") " pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:47.756064 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.756033 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lss5w\" (UniqueName: \"kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w\") pod \"authorino-5d4b67c446-cxxxw\" (UID: \"5c3e80c7-97db-4e6b-a9d5-377a806e9978\") " pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:47.773952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.773930 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lss5w\" (UniqueName: \"kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w\") pod \"authorino-5d4b67c446-cxxxw\" (UID: \"5c3e80c7-97db-4e6b-a9d5-377a806e9978\") " pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:47.883253 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.883215 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:47.891363 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.891335 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:47.925818 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.925777 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:47.928157 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.928133 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:47.943349 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.943322 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:47.957977 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:47.957947 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mflz7\" (UniqueName: \"kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7\") pod \"authorino-b5bdb675c-kpp4g\" (UID: \"830dc259-4a8a-47c7-9c6c-b33674b92cb4\") " pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:48.038643 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.038614 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:48.040208 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:33:48.040174 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c3e80c7_97db_4e6b_a9d5_377a806e9978.slice/crio-ba28b5e0f49d53e98b4a34b971bbf5c801657f45caf28e79bd1cad5ec8d9b6eb WatchSource:0}: Error finding container ba28b5e0f49d53e98b4a34b971bbf5c801657f45caf28e79bd1cad5ec8d9b6eb: Status 404 returned error can't find the container with id ba28b5e0f49d53e98b4a34b971bbf5c801657f45caf28e79bd1cad5ec8d9b6eb Jul 22 11:33:48.058986 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.058958 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mflz7\" (UniqueName: \"kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7\") pod \"authorino-b5bdb675c-kpp4g\" (UID: \"830dc259-4a8a-47c7-9c6c-b33674b92cb4\") " pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:48.068405 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.068376 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mflz7\" (UniqueName: \"kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7\") pod \"authorino-b5bdb675c-kpp4g\" (UID: \"830dc259-4a8a-47c7-9c6c-b33674b92cb4\") " pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:48.182965 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.182881 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:48.183123 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.183112 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:48.216152 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.216116 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:33:48.218980 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.218953 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.221609 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.221568 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 11:33:48.232334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.232310 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:33:48.260356 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.260325 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j584p\" (UniqueName: \"kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.260499 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.260372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.331660 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.331488 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:48.333800 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:33:48.333772 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod830dc259_4a8a_47c7_9c6c_b33674b92cb4.slice/crio-eaa927ea988a679a77d99d597f5db3dcbf551900e720c54426f5a7e48b5423c3 WatchSource:0}: Error finding container eaa927ea988a679a77d99d597f5db3dcbf551900e720c54426f5a7e48b5423c3: Status 404 returned error can't find the container with id eaa927ea988a679a77d99d597f5db3dcbf551900e720c54426f5a7e48b5423c3 Jul 22 11:33:48.361644 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.361613 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j584p\" (UniqueName: \"kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.361748 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.361674 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.364077 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.364060 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.374113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.374095 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j584p\" (UniqueName: \"kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p\") pod \"authorino-74648d77bf-lgtnd\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.528377 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.528356 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:33:48.678557 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.678524 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:33:48.682059 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:33:48.682035 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c0661d7_0c7c_4c8a_98c4_a8243ddad8a5.slice/crio-4eb59d81c56e869f8b2bd2d9aff41a8eee90cb29971fe2de137ad55173ef9490 WatchSource:0}: Error finding container 4eb59d81c56e869f8b2bd2d9aff41a8eee90cb29971fe2de137ad55173ef9490: Status 404 returned error can't find the container with id 4eb59d81c56e869f8b2bd2d9aff41a8eee90cb29971fe2de137ad55173ef9490 Jul 22 11:33:48.760715 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.760679 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" event={"ID":"830dc259-4a8a-47c7-9c6c-b33674b92cb4","Type":"ContainerStarted","Data":"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4"} Jul 22 11:33:48.760890 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.760723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" event={"ID":"830dc259-4a8a-47c7-9c6c-b33674b92cb4","Type":"ContainerStarted","Data":"eaa927ea988a679a77d99d597f5db3dcbf551900e720c54426f5a7e48b5423c3"} Jul 22 11:33:48.760890 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.760716 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" podUID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" containerName="authorino" containerID="cri-o://1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4" gracePeriod=30 Jul 22 11:33:48.762038 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.762008 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74648d77bf-lgtnd" event={"ID":"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5","Type":"ContainerStarted","Data":"4eb59d81c56e869f8b2bd2d9aff41a8eee90cb29971fe2de137ad55173ef9490"} Jul 22 11:33:48.763651 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.763579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" event={"ID":"5c3e80c7-97db-4e6b-a9d5-377a806e9978","Type":"ContainerStarted","Data":"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024"} Jul 22 11:33:48.763651 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.763628 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" podUID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" containerName="authorino" containerID="cri-o://88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024" gracePeriod=30 Jul 22 11:33:48.763651 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.763639 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" event={"ID":"5c3e80c7-97db-4e6b-a9d5-377a806e9978","Type":"ContainerStarted","Data":"ba28b5e0f49d53e98b4a34b971bbf5c801657f45caf28e79bd1cad5ec8d9b6eb"} Jul 22 11:33:48.797038 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.796943 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" podStartSLOduration=1.462843672 podStartE2EDuration="1.796926175s" totalImagesPullingTime="334.082503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:33:48.79590241 +0000 UTC m=+597.303803298" watchObservedRunningTime="2026-07-22 11:33:48.796926175 +0000 UTC m=+597.304827060" Jul 22 11:33:48.822391 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:48.822336 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" podStartSLOduration=1.33635218 podStartE2EDuration="1.822316845s" totalImagesPullingTime="485.964665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:33:48.820662631 +0000 UTC m=+597.328563518" watchObservedRunningTime="2026-07-22 11:33:48.822316845 +0000 UTC m=+597.330217731" Jul 22 11:33:49.027964 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.027939 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:49.031222 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.031203 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:49.066759 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.066652 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mflz7\" (UniqueName: \"kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7\") pod \"830dc259-4a8a-47c7-9c6c-b33674b92cb4\" (UID: \"830dc259-4a8a-47c7-9c6c-b33674b92cb4\") " Jul 22 11:33:49.066961 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.066752 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lss5w\" (UniqueName: \"kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w\") pod \"5c3e80c7-97db-4e6b-a9d5-377a806e9978\" (UID: \"5c3e80c7-97db-4e6b-a9d5-377a806e9978\") " Jul 22 11:33:49.069656 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.069581 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7" (OuterVolumeSpecName: "kube-api-access-mflz7") pod "830dc259-4a8a-47c7-9c6c-b33674b92cb4" (UID: "830dc259-4a8a-47c7-9c6c-b33674b92cb4"). InnerVolumeSpecName "kube-api-access-mflz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:33:49.069791 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.069680 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w" (OuterVolumeSpecName: "kube-api-access-lss5w") pod "5c3e80c7-97db-4e6b-a9d5-377a806e9978" (UID: "5c3e80c7-97db-4e6b-a9d5-377a806e9978"). InnerVolumeSpecName "kube-api-access-lss5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:33:49.167888 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.167858 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mflz7\" (UniqueName: \"kubernetes.io/projected/830dc259-4a8a-47c7-9c6c-b33674b92cb4-kube-api-access-mflz7\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:33:49.167888 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.167886 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lss5w\" (UniqueName: \"kubernetes.io/projected/5c3e80c7-97db-4e6b-a9d5-377a806e9978-kube-api-access-lss5w\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:33:49.770206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.770119 2577 generic.go:356] "Generic (PLEG): container finished" podID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" containerID="1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4" exitCode=0 Jul 22 11:33:49.770206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.770164 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" Jul 22 11:33:49.770206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.770176 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" event={"ID":"830dc259-4a8a-47c7-9c6c-b33674b92cb4","Type":"ContainerDied","Data":"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4"} Jul 22 11:33:49.770790 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.770220 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b5bdb675c-kpp4g" event={"ID":"830dc259-4a8a-47c7-9c6c-b33674b92cb4","Type":"ContainerDied","Data":"eaa927ea988a679a77d99d597f5db3dcbf551900e720c54426f5a7e48b5423c3"} Jul 22 11:33:49.770790 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.770239 2577 scope.go:117] "RemoveContainer" containerID="1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4" Jul 22 11:33:49.771856 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.771809 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74648d77bf-lgtnd" event={"ID":"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5","Type":"ContainerStarted","Data":"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f"} Jul 22 11:33:49.773150 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.773131 2577 generic.go:356] "Generic (PLEG): container finished" podID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" containerID="88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024" exitCode=0 Jul 22 11:33:49.773274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.773187 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" Jul 22 11:33:49.773274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.773221 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" event={"ID":"5c3e80c7-97db-4e6b-a9d5-377a806e9978","Type":"ContainerDied","Data":"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024"} Jul 22 11:33:49.773274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.773260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-cxxxw" event={"ID":"5c3e80c7-97db-4e6b-a9d5-377a806e9978","Type":"ContainerDied","Data":"ba28b5e0f49d53e98b4a34b971bbf5c801657f45caf28e79bd1cad5ec8d9b6eb"} Jul 22 11:33:49.780085 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.780065 2577 scope.go:117] "RemoveContainer" containerID="1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4" Jul 22 11:33:49.780336 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:33:49.780318 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4\": container with ID starting with 1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4 not found: ID does not exist" containerID="1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4" Jul 22 11:33:49.780394 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.780345 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4"} err="failed to get container status \"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4\": rpc error: code = NotFound desc = could not find container \"1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4\": container with ID starting with 1bdf9eb5451eff2c3a07aea1ef4a35cb48ca076dd0e321254f34e5eefa0febd4 not found: ID does not exist" Jul 22 11:33:49.780394 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.780363 2577 scope.go:117] "RemoveContainer" containerID="88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024" Jul 22 11:33:49.788354 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.788336 2577 scope.go:117] "RemoveContainer" containerID="88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024" Jul 22 11:33:49.788632 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:33:49.788586 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024\": container with ID starting with 88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024 not found: ID does not exist" containerID="88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024" Jul 22 11:33:49.788720 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.788640 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024"} err="failed to get container status \"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024\": rpc error: code = NotFound desc = could not find container \"88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024\": container with ID starting with 88202df38a45134207202a3f85a654d2894f7668cf292d1af231a9748c51f024 not found: ID does not exist" Jul 22 11:33:49.797860 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.797811 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74648d77bf-lgtnd" podStartSLOduration=1.1258178 podStartE2EDuration="1.797794373s" totalImagesPullingTime="671.976573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:33:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:33:49.796975253 +0000 UTC m=+598.304876139" watchObservedRunningTime="2026-07-22 11:33:49.797794373 +0000 UTC m=+598.305695260" Jul 22 11:33:49.820360 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.820290 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:49.824794 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.824768 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-cxxxw"] Jul 22 11:33:49.837777 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.837745 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:49.837931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.837913 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-8ltrb" podUID="c001157a-05d7-4f57-bbd4-12fc3a551afa" containerName="authorino" containerID="cri-o://86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8" gracePeriod=30 Jul 22 11:33:49.843036 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.843008 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:49.854631 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:49.854589 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b5bdb675c-kpp4g"] Jul 22 11:33:50.067909 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.067834 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" path="/var/lib/kubelet/pods/5c3e80c7-97db-4e6b-a9d5-377a806e9978/volumes" Jul 22 11:33:50.068161 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.068148 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" path="/var/lib/kubelet/pods/830dc259-4a8a-47c7-9c6c-b33674b92cb4/volumes" Jul 22 11:33:50.110150 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.110124 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:50.178416 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.178381 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgj9r\" (UniqueName: \"kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r\") pod \"c001157a-05d7-4f57-bbd4-12fc3a551afa\" (UID: \"c001157a-05d7-4f57-bbd4-12fc3a551afa\") " Jul 22 11:33:50.180683 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.180638 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r" (OuterVolumeSpecName: "kube-api-access-dgj9r") pod "c001157a-05d7-4f57-bbd4-12fc3a551afa" (UID: "c001157a-05d7-4f57-bbd4-12fc3a551afa"). InnerVolumeSpecName "kube-api-access-dgj9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:33:50.279889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.279854 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgj9r\" (UniqueName: \"kubernetes.io/projected/c001157a-05d7-4f57-bbd4-12fc3a551afa-kube-api-access-dgj9r\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:33:50.781994 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.781950 2577 generic.go:356] "Generic (PLEG): container finished" podID="c001157a-05d7-4f57-bbd4-12fc3a551afa" containerID="86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8" exitCode=0 Jul 22 11:33:50.782474 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.782009 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-8ltrb" Jul 22 11:33:50.782474 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.782032 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8ltrb" event={"ID":"c001157a-05d7-4f57-bbd4-12fc3a551afa","Type":"ContainerDied","Data":"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8"} Jul 22 11:33:50.782474 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.782084 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-8ltrb" event={"ID":"c001157a-05d7-4f57-bbd4-12fc3a551afa","Type":"ContainerDied","Data":"65126e287d8a9d5b1adbbcaf746ba602c0d431923f304d1c48922f7468bc9a8e"} Jul 22 11:33:50.782474 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.782109 2577 scope.go:117] "RemoveContainer" containerID="86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8" Jul 22 11:33:50.791182 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.791163 2577 scope.go:117] "RemoveContainer" containerID="86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8" Jul 22 11:33:50.791457 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:33:50.791434 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8\": container with ID starting with 86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8 not found: ID does not exist" containerID="86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8" Jul 22 11:33:50.791514 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.791470 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8"} err="failed to get container status \"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8\": rpc error: code = NotFound desc = could not find container \"86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8\": container with ID starting with 86e77b91fd4262ad18d524b01e2200e21874805ec85fde856ef03d71acd1d3f8 not found: ID does not exist" Jul 22 11:33:50.813537 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.813506 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:50.825205 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:50.825176 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-8ltrb"] Jul 22 11:33:51.928060 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:51.928030 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:33:51.930815 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:51.930795 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:33:52.067675 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:33:52.067645 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c001157a-05d7-4f57-bbd4-12fc3a551afa" path="/var/lib/kubelet/pods/c001157a-05d7-4f57-bbd4-12fc3a551afa/volumes" Jul 22 11:34:59.895215 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895173 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn"] Jul 22 11:34:59.895762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895691 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" containerName="authorino" Jul 22 11:34:59.895762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895712 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" containerName="authorino" Jul 22 11:34:59.895762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895726 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c001157a-05d7-4f57-bbd4-12fc3a551afa" containerName="authorino" Jul 22 11:34:59.895762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895735 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c001157a-05d7-4f57-bbd4-12fc3a551afa" containerName="authorino" Jul 22 11:34:59.895762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895760 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" containerName="authorino" Jul 22 11:34:59.896028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895768 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" containerName="authorino" Jul 22 11:34:59.896028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895843 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c001157a-05d7-4f57-bbd4-12fc3a551afa" containerName="authorino" Jul 22 11:34:59.896028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895853 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c3e80c7-97db-4e6b-a9d5-377a806e9978" containerName="authorino" Jul 22 11:34:59.896028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.895860 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="830dc259-4a8a-47c7-9c6c-b33674b92cb4" containerName="authorino" Jul 22 11:34:59.897690 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.897669 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.900212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.900184 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 11:34:59.900829 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.900806 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 11:34:59.900977 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.900861 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 11:34:59.900977 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.900950 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-t5vvq\"" Jul 22 11:34:59.910076 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.910054 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn"] Jul 22 11:34:59.984536 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984504 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.984706 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984545 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.984706 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984567 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmn2n\" (UniqueName: \"kubernetes.io/projected/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-kube-api-access-dmn2n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.984706 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984672 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.984810 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984720 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:34:59.984810 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:34:59.984746 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.085821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085792 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.085821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085826 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmn2n\" (UniqueName: \"kubernetes.io/projected/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-kube-api-access-dmn2n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086048 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086048 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085913 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086048 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085932 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086048 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.085973 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086246 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.086231 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086311 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.086293 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.086372 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.086352 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.088424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.088395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.088647 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.088629 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.095758 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.095738 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmn2n\" (UniqueName: \"kubernetes.io/projected/3b90ea5c-1d1b-467c-864a-d9029c4a8b45-kube-api-access-dmn2n\") pod \"e2e-distinct-simulated-kserve-59975d69f9-k99rn\" (UID: \"3b90ea5c-1d1b-467c-864a-d9029c4a8b45\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.208877 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.208788 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:00.344876 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.344850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn"] Jul 22 11:35:00.346486 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:35:00.346455 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b90ea5c_1d1b_467c_864a_d9029c4a8b45.slice/crio-504884b747bfb1ec0a7a669dd62df3b827099942abb0dc1f56ba4e2445e3cf2e WatchSource:0}: Error finding container 504884b747bfb1ec0a7a669dd62df3b827099942abb0dc1f56ba4e2445e3cf2e: Status 404 returned error can't find the container with id 504884b747bfb1ec0a7a669dd62df3b827099942abb0dc1f56ba4e2445e3cf2e Jul 22 11:35:00.347730 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:00.347713 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:35:01.131392 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:01.131356 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" event={"ID":"3b90ea5c-1d1b-467c-864a-d9029c4a8b45","Type":"ContainerStarted","Data":"504884b747bfb1ec0a7a669dd62df3b827099942abb0dc1f56ba4e2445e3cf2e"} Jul 22 11:35:02.137500 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:02.137455 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" event={"ID":"3b90ea5c-1d1b-467c-864a-d9029c4a8b45","Type":"ContainerStarted","Data":"9ab0b1ae823b21cb8ccd667f3763d33dd6e3adaebf7e3e1abd8ee8885a86b9ba"} Jul 22 11:35:02.158082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:02.158035 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" podStartSLOduration=1.9881344429999999 podStartE2EDuration="3.15801898s" totalImagesPullingTime="1.169884537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:34:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:35:02.15649419 +0000 UTC m=+670.664395075" watchObservedRunningTime="2026-07-22 11:35:02.15801898 +0000 UTC m=+670.665919907" Jul 22 11:35:10.209128 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:10.209096 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:10.209567 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:10.209238 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:10.221992 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:10.221970 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:35:11.194612 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:35:11.194568 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-k99rn" Jul 22 11:37:14.721450 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.721410 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77fcc66564-6ww8x"] Jul 22 11:37:14.724042 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.724022 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.735567 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.735543 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77fcc66564-6ww8x"] Jul 22 11:37:14.862896 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.862860 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbkpw\" (UniqueName: \"kubernetes.io/projected/49bb470b-9c19-4a41-92dc-1ef58f7e2928-kube-api-access-lbkpw\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.862896 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.862909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bb470b-9c19-4a41-92dc-1ef58f7e2928-tls-cert\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.963787 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.963753 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbkpw\" (UniqueName: \"kubernetes.io/projected/49bb470b-9c19-4a41-92dc-1ef58f7e2928-kube-api-access-lbkpw\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.963958 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.963806 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bb470b-9c19-4a41-92dc-1ef58f7e2928-tls-cert\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.966392 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.966370 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/49bb470b-9c19-4a41-92dc-1ef58f7e2928-tls-cert\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:14.981917 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:14.981847 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbkpw\" (UniqueName: \"kubernetes.io/projected/49bb470b-9c19-4a41-92dc-1ef58f7e2928-kube-api-access-lbkpw\") pod \"authorino-77fcc66564-6ww8x\" (UID: \"49bb470b-9c19-4a41-92dc-1ef58f7e2928\") " pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:15.033035 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.033001 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77fcc66564-6ww8x" Jul 22 11:37:15.183715 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.183685 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77fcc66564-6ww8x"] Jul 22 11:37:15.185268 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:37:15.185232 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49bb470b_9c19_4a41_92dc_1ef58f7e2928.slice/crio-c391d3026e28e65756fa970543c1c8b5c14fdddfad68256b9a960a5a2921d073 WatchSource:0}: Error finding container c391d3026e28e65756fa970543c1c8b5c14fdddfad68256b9a960a5a2921d073: Status 404 returned error can't find the container with id c391d3026e28e65756fa970543c1c8b5c14fdddfad68256b9a960a5a2921d073 Jul 22 11:37:15.806372 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.806333 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77fcc66564-6ww8x" event={"ID":"49bb470b-9c19-4a41-92dc-1ef58f7e2928","Type":"ContainerStarted","Data":"3f0c923348a4577bbe44d9e2869a2b8bf5baf3f792253db9fa4671d82a44936a"} Jul 22 11:37:15.806372 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.806379 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-77fcc66564-6ww8x" event={"ID":"49bb470b-9c19-4a41-92dc-1ef58f7e2928","Type":"ContainerStarted","Data":"c391d3026e28e65756fa970543c1c8b5c14fdddfad68256b9a960a5a2921d073"} Jul 22 11:37:15.824002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.823952 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-77fcc66564-6ww8x" podStartSLOduration=1.372260796 podStartE2EDuration="1.823937993s" totalImagesPullingTime="451.677197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:37:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:37:15.822806734 +0000 UTC m=+804.330707621" watchObservedRunningTime="2026-07-22 11:37:15.823937993 +0000 UTC m=+804.331838883" Jul 22 11:37:15.892461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.892367 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:37:15.892650 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:15.892543 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-74648d77bf-lgtnd" podUID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" containerName="authorino" containerID="cri-o://a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f" gracePeriod=30 Jul 22 11:37:16.158176 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.157936 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:37:16.275687 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.275656 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j584p\" (UniqueName: \"kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p\") pod \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " Jul 22 11:37:16.275887 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.275725 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert\") pod \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\" (UID: \"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5\") " Jul 22 11:37:16.278441 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.278404 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p" (OuterVolumeSpecName: "kube-api-access-j584p") pod "1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" (UID: "1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5"). InnerVolumeSpecName "kube-api-access-j584p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:37:16.287005 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.286973 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" (UID: "1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:37:16.376843 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.376812 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-tls-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:37:16.376843 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.376840 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j584p\" (UniqueName: \"kubernetes.io/projected/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5-kube-api-access-j584p\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:37:16.816616 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.816552 2577 generic.go:356] "Generic (PLEG): container finished" podID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" containerID="a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f" exitCode=0 Jul 22 11:37:16.817045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.816644 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74648d77bf-lgtnd" Jul 22 11:37:16.817045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.816637 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74648d77bf-lgtnd" event={"ID":"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5","Type":"ContainerDied","Data":"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f"} Jul 22 11:37:16.817045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.816765 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74648d77bf-lgtnd" event={"ID":"1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5","Type":"ContainerDied","Data":"4eb59d81c56e869f8b2bd2d9aff41a8eee90cb29971fe2de137ad55173ef9490"} Jul 22 11:37:16.817045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.816788 2577 scope.go:117] "RemoveContainer" containerID="a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f" Jul 22 11:37:16.826972 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.826952 2577 scope.go:117] "RemoveContainer" containerID="a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f" Jul 22 11:37:16.827231 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:37:16.827209 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f\": container with ID starting with a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f not found: ID does not exist" containerID="a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f" Jul 22 11:37:16.827276 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.827239 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f"} err="failed to get container status \"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f\": rpc error: code = NotFound desc = could not find container \"a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f\": container with ID starting with a31ed5609fb85ef50f6a6144a9b16d896629971139d00e320c5fee030d9bf07f not found: ID does not exist" Jul 22 11:37:16.844109 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.844079 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:37:16.851465 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:16.851435 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-74648d77bf-lgtnd"] Jul 22 11:37:18.067386 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:37:18.067353 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" path="/var/lib/kubelet/pods/1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5/volumes" Jul 22 11:38:51.962288 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:38:51.962204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:38:51.962802 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:38:51.962470 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:43:51.998188 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:43:51.998158 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:43:51.999751 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:43:51.999731 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:47:54.211843 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.211761 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:47:54.212325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.212079 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" containerName="authorino" Jul 22 11:47:54.212325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.212090 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" containerName="authorino" Jul 22 11:47:54.212325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.212160 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c0661d7-0c7c-4c8a-98c4-a8243ddad8a5" containerName="authorino" Jul 22 11:47:54.215210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.215191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.217435 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.217416 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ff5dcb7e-kserve-self-signed-certs\"" Jul 22 11:47:54.229428 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.229403 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:47:54.297104 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297073 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.297257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297117 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.297257 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297139 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wn4q\" (UniqueName: \"kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.297334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297253 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.297334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.297334 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.297311 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398102 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398059 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398102 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398103 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398170 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wn4q\" (UniqueName: \"kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398574 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398527 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398574 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398566 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.398771 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.398687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.400608 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.400574 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.400886 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.400867 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.407701 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.407675 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wn4q\" (UniqueName: \"kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q\") pod \"e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.524737 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.524636 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:47:54.661964 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.661940 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:47:54.664177 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:47:54.664134 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15e7209c_b788_4604_b861_9436302cb09d.slice/crio-ca3d5e6e6025e00255a93f7618fbee0143c1fd543e49633b0e7a29b3fad1a30d WatchSource:0}: Error finding container ca3d5e6e6025e00255a93f7618fbee0143c1fd543e49633b0e7a29b3fad1a30d: Status 404 returned error can't find the container with id ca3d5e6e6025e00255a93f7618fbee0143c1fd543e49633b0e7a29b3fad1a30d Jul 22 11:47:54.949919 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.949885 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" event={"ID":"15e7209c-b788-4604-b861-9436302cb09d","Type":"ContainerStarted","Data":"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5"} Jul 22 11:47:54.950094 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.949935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" event={"ID":"15e7209c-b788-4604-b861-9436302cb09d","Type":"ContainerStarted","Data":"ca3d5e6e6025e00255a93f7618fbee0143c1fd543e49633b0e7a29b3fad1a30d"} Jul 22 11:47:54.968654 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:47:54.968581 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" podStartSLOduration=0.968564647 podStartE2EDuration="968.564647ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:47:54.96708876 +0000 UTC m=+1443.474989637" watchObservedRunningTime="2026-07-22 11:47:54.968564647 +0000 UTC m=+1443.476465532" Jul 22 11:48:04.525003 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:04.524919 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:04.525003 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:04.524967 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:04.537322 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:04.537296 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:05.012516 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:05.012484 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:36.867515 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:36.867482 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:48:51.888663 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:51.888620 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" podUID="15e7209c-b788-4604-b861-9436302cb09d" containerName="main" containerID="cri-o://2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5" gracePeriod=45 Jul 22 11:48:52.033401 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.033374 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:48:52.038182 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.038158 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:48:52.149091 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.149029 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:52.229640 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.229581 2577 generic.go:356] "Generic (PLEG): container finished" podID="15e7209c-b788-4604-b861-9436302cb09d" containerID="2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5" exitCode=0 Jul 22 11:48:52.229640 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.229636 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" Jul 22 11:48:52.229640 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.229644 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" event={"ID":"15e7209c-b788-4604-b861-9436302cb09d","Type":"ContainerDied","Data":"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5"} Jul 22 11:48:52.229918 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.229683 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp" event={"ID":"15e7209c-b788-4604-b861-9436302cb09d","Type":"ContainerDied","Data":"ca3d5e6e6025e00255a93f7618fbee0143c1fd543e49633b0e7a29b3fad1a30d"} Jul 22 11:48:52.229918 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.229703 2577 scope.go:117] "RemoveContainer" containerID="2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5" Jul 22 11:48:52.238391 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.238373 2577 scope.go:117] "RemoveContainer" containerID="2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5" Jul 22 11:48:52.238674 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:48:52.238646 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5\": container with ID starting with 2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5 not found: ID does not exist" containerID="2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5" Jul 22 11:48:52.238777 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.238682 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5"} err="failed to get container status \"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5\": rpc error: code = NotFound desc = could not find container \"2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5\": container with ID starting with 2566cfda268280c566ba9b21e045ae42016f7c7429237189c0b48b7dbc943aa5 not found: ID does not exist" Jul 22 11:48:52.298688 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.298654 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.298855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.298749 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wn4q\" (UniqueName: \"kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.298855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.298786 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.298855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.298830 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.299036 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299014 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.299095 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299055 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir\") pod \"15e7209c-b788-4604-b861-9436302cb09d\" (UID: \"15e7209c-b788-4604-b861-9436302cb09d\") " Jul 22 11:48:52.299095 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299060 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home" (OuterVolumeSpecName: "home") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:48:52.299312 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299271 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache" (OuterVolumeSpecName: "model-cache") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:48:52.299432 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299315 2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-home\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.299432 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.299370 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:48:52.301223 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.301199 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q" (OuterVolumeSpecName: "kube-api-access-8wn4q") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "kube-api-access-8wn4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:48:52.301370 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.301282 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 11:48:52.301434 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.301377 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm" (OuterVolumeSpecName: "dshm") pod "15e7209c-b788-4604-b861-9436302cb09d" (UID: "15e7209c-b788-4604-b861-9436302cb09d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:48:52.400012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.399929 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wn4q\" (UniqueName: \"kubernetes.io/projected/15e7209c-b788-4604-b861-9436302cb09d-kube-api-access-8wn4q\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.400012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.399956 2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-model-cache\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.400012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.399967 2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-dshm\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.400012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.399975 2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/15e7209c-b788-4604-b861-9436302cb09d-tmp-dir\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.400012 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.399983 2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/15e7209c-b788-4604-b861-9436302cb09d-tls-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:48:52.553603 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.553565 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:48:52.558275 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:52.558249 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ff5dcb7e-kserve-7c6d4777c9-zv9kp"] Jul 22 11:48:54.067453 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:48:54.067407 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15e7209c-b788-4604-b861-9436302cb09d" path="/var/lib/kubelet/pods/15e7209c-b788-4604-b861-9436302cb09d/volumes" Jul 22 11:52:49.143570 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.143533 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw"] Jul 22 11:52:49.144004 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.143875 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15e7209c-b788-4604-b861-9436302cb09d" containerName="main" Jul 22 11:52:49.144004 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.143886 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="15e7209c-b788-4604-b861-9436302cb09d" containerName="main" Jul 22 11:52:49.144004 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.143945 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="15e7209c-b788-4604-b861-9436302cb09d" containerName="main" Jul 22 11:52:49.146826 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.146807 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:49.149029 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.149007 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 11:52:49.149115 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.149026 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m4wzf\"" Jul 22 11:52:49.155327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.155300 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw"] Jul 22 11:52:49.255172 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.255137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw\" (UID: \"1671d9bc-b21e-4b45-a821-12409edbb1a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:49.356300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.356253 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw\" (UID: \"1671d9bc-b21e-4b45-a821-12409edbb1a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:49.356955 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.356936 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw\" (UID: \"1671d9bc-b21e-4b45-a821-12409edbb1a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:49.458687 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.458611 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:49.605402 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.605376 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw"] Jul 22 11:52:49.607937 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:52:49.607911 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1671d9bc_b21e_4b45_a821_12409edbb1a4.slice/crio-c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668 WatchSource:0}: Error finding container c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668: Status 404 returned error can't find the container with id c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668 Jul 22 11:52:49.609818 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:49.609799 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 11:52:50.412193 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:50.412157 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" event={"ID":"1671d9bc-b21e-4b45-a821-12409edbb1a4","Type":"ContainerStarted","Data":"c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668"} Jul 22 11:52:53.430327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:53.430289 2577 generic.go:356] "Generic (PLEG): container finished" podID="1671d9bc-b21e-4b45-a821-12409edbb1a4" containerID="a155411d968c5165686502155e3c908c09d37b7ce894c005dd5c7d1ce80a98ab" exitCode=0 Jul 22 11:52:53.430718 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:53.430374 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" event={"ID":"1671d9bc-b21e-4b45-a821-12409edbb1a4","Type":"ContainerDied","Data":"a155411d968c5165686502155e3c908c09d37b7ce894c005dd5c7d1ce80a98ab"} Jul 22 11:52:54.566148 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:54.566119 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:54.705080 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:54.704988 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca\") pod \"1671d9bc-b21e-4b45-a821-12409edbb1a4\" (UID: \"1671d9bc-b21e-4b45-a821-12409edbb1a4\") " Jul 22 11:52:54.705330 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:54.705308 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1671d9bc-b21e-4b45-a821-12409edbb1a4" (UID: "1671d9bc-b21e-4b45-a821-12409edbb1a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:52:54.806051 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:54.806002 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1671d9bc-b21e-4b45-a821-12409edbb1a4-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:52:55.441824 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:55.441787 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" event={"ID":"1671d9bc-b21e-4b45-a821-12409edbb1a4","Type":"ContainerDied","Data":"c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668"} Jul 22 11:52:55.441824 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:55.441805 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw" Jul 22 11:52:55.441824 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:52:55.441835 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c87fbe35e17e792898201084774875492e7f47598fd27e424bf7c33534624668" Jul 22 11:53:04.238205 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:04.238164 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw"] Jul 22 11:53:04.240507 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:04.240479 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8faa1d52-h48fw"] Jul 22 11:53:05.434700 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.434666 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:05.435133 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.434999 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1671d9bc-b21e-4b45-a821-12409edbb1a4" containerName="revoke-keys" Jul 22 11:53:05.435133 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.435011 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1671d9bc-b21e-4b45-a821-12409edbb1a4" containerName="revoke-keys" Jul 22 11:53:05.435133 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.435072 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1671d9bc-b21e-4b45-a821-12409edbb1a4" containerName="revoke-keys" Jul 22 11:53:05.438901 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.438878 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.441289 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.441259 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d8c0f9e9-gw-openshift-default-dockercfg-ccxpn\"" Jul 22 11:53:05.441289 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.441264 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:53:05.453581 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.453555 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:05.599563 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599528 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlcrc\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599581 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599659 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599721 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599766 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599754 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599956 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599794 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599956 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599835 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.599956 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.599863 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.700988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701048 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701068 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701086 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701090 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701101 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701157 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701182 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701526 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701230 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlcrc\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701544 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701589 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701817 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701680 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701958 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.701995 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.701966 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.703629 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.703587 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.703771 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.703752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.709841 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.709811 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.709948 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.709893 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlcrc\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc\") pod \"e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.749515 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.749483 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:05.894326 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:05.894295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:05.896295 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:53:05.896261 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408fca5d_71eb_4706_b108_5c8326d9588f.slice/crio-252dd130aff7d5a7b51b0399f4a69eaca97863cb8273c6fed145a231f1b09f0a WatchSource:0}: Error finding container 252dd130aff7d5a7b51b0399f4a69eaca97863cb8273c6fed145a231f1b09f0a: Status 404 returned error can't find the container with id 252dd130aff7d5a7b51b0399f4a69eaca97863cb8273c6fed145a231f1b09f0a Jul 22 11:53:06.067327 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:06.067246 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1671d9bc-b21e-4b45-a821-12409edbb1a4" path="/var/lib/kubelet/pods/1671d9bc-b21e-4b45-a821-12409edbb1a4/volumes" Jul 22 11:53:06.495978 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:06.495946 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerStarted","Data":"252dd130aff7d5a7b51b0399f4a69eaca97863cb8273c6fed145a231f1b09f0a"} Jul 22 11:53:08.480794 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:08.480757 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:08.481081 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:08.480832 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:08.481081 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:08.480864 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:09.514223 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:09.514190 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerStarted","Data":"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711"} Jul 22 11:53:09.536173 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:09.536124 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podStartSLOduration=1.954219524 podStartE2EDuration="4.536109067s" totalImagesPullingTime="2.581889543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:53:09.535081541 +0000 UTC m=+1758.042982427" watchObservedRunningTime="2026-07-22 11:53:09.536109067 +0000 UTC m=+1758.044009953" Jul 22 11:53:09.750075 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:09.750037 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:09.755303 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:09.755277 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:10.518990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:10.518959 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:10.520023 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:10.520004 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:22.579757 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.579723 2577 generic.go:356] "Generic (PLEG): container finished" podID="408fca5d-71eb-4706-b108-5c8326d9588f" containerID="9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711" exitCode=0 Jul 22 11:53:22.580155 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.579780 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerDied","Data":"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711"} Jul 22 11:53:22.580155 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.580132 2577 scope.go:117] "RemoveContainer" containerID="9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711" Jul 22 11:53:22.582103 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.582065 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:22.582199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.582148 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:22.582199 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:22.582182 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:53:23.590149 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:23.590109 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerStarted","Data":"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7"} Jul 22 11:53:23.750275 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:23.750236 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:23.751321 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:23.751300 2577 patch_prober.go:28] interesting pod/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:53:23.751373 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:23.751347 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 22 11:53:24.750534 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:24.750498 2577 patch_prober.go:28] interesting pod/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:53:24.750954 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:24.750556 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 22 11:53:25.519156 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:25.519116 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:25.749955 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:25.749915 2577 patch_prober.go:28] interesting pod/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:53:25.750136 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:25.749976 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 22 11:53:26.749928 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:26.749883 2577 patch_prober.go:28] interesting pod/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body= Jul 22 11:53:26.750410 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:26.749948 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" Jul 22 11:53:27.754048 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:27.754014 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:27.754945 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:27.754925 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:51.381207 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:51.381163 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:51.381658 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:51.381386 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" containerID="cri-o://d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7" gracePeriod=30 Jul 22 11:53:52.070861 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:52.070836 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:53:52.081346 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:52.081317 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:53:56.625968 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.625944 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:56.664762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664680 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664762 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664726 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664768 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664809 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664848 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664873 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664889 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664907 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.664991 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.664936 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlcrc\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc\") pod \"408fca5d-71eb-4706-b108-5c8326d9588f\" (UID: \"408fca5d-71eb-4706-b108-5c8326d9588f\") " Jul 22 11:53:56.665365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.665058 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:56.665365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.665237 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.665365 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.665353 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:53:56.665532 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.665425 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:56.665532 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.665479 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:56.666423 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.666386 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data" (OuterVolumeSpecName: "istio-data") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:56.667576 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.667538 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 11:53:56.667725 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.667630 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token" (OuterVolumeSpecName: "istio-token") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:53:56.667725 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.667712 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc" (OuterVolumeSpecName: "kube-api-access-hlcrc") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "kube-api-access-hlcrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:53:56.667859 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.667746 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "408fca5d-71eb-4706-b108-5c8326d9588f" (UID: "408fca5d-71eb-4706-b108-5c8326d9588f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:53:56.761859 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.761820 2577 generic.go:356] "Generic (PLEG): container finished" podID="408fca5d-71eb-4706-b108-5c8326d9588f" containerID="d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7" exitCode=0 Jul 22 11:53:56.762022 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.761874 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" Jul 22 11:53:56.762022 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.761877 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerDied","Data":"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7"} Jul 22 11:53:56.762022 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.761931 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k" event={"ID":"408fca5d-71eb-4706-b108-5c8326d9588f","Type":"ContainerDied","Data":"252dd130aff7d5a7b51b0399f4a69eaca97863cb8273c6fed145a231f1b09f0a"} Jul 22 11:53:56.762022 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.761963 2577 scope.go:117] "RemoveContainer" containerID="d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7" Jul 22 11:53:56.765680 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765654 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/408fca5d-71eb-4706-b108-5c8326d9588f-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765680 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765678 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765689 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765702 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765711 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/408fca5d-71eb-4706-b108-5c8326d9588f-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765719 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765728 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlcrc\" (UniqueName: \"kubernetes.io/projected/408fca5d-71eb-4706-b108-5c8326d9588f-kube-api-access-hlcrc\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.765855 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.765735 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/408fca5d-71eb-4706-b108-5c8326d9588f-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:53:56.771368 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.771351 2577 scope.go:117] "RemoveContainer" containerID="9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711" Jul 22 11:53:56.779790 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.779772 2577 scope.go:117] "RemoveContainer" containerID="d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7" Jul 22 11:53:56.780056 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:53:56.780034 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7\": container with ID starting with d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7 not found: ID does not exist" containerID="d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7" Jul 22 11:53:56.780129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.780069 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7"} err="failed to get container status \"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7\": rpc error: code = NotFound desc = could not find container \"d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7\": container with ID starting with d9c90fc8c7c1e949dee033dd519276d6f5af017bb78c8340dd5b43bebad0b7f7 not found: ID does not exist" Jul 22 11:53:56.780129 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.780095 2577 scope.go:117] "RemoveContainer" containerID="9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711" Jul 22 11:53:56.780376 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:53:56.780343 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711\": container with ID starting with 9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711 not found: ID does not exist" containerID="9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711" Jul 22 11:53:56.780419 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.780386 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711"} err="failed to get container status \"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711\": rpc error: code = NotFound desc = could not find container \"9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711\": container with ID starting with 9721e603bf2755cf7d4ca54a3d79bde801e85c61d50bd2afb13cb917454a2711 not found: ID does not exist" Jul 22 11:53:56.789849 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.789817 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:56.798002 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:56.797981 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d8c0f9e9-gw-openshift-default-7c44b7c6f5-bqt8k"] Jul 22 11:53:58.070853 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:53:58.070816 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" path="/var/lib/kubelet/pods/408fca5d-71eb-4706-b108-5c8326d9588f/volumes" Jul 22 11:54:20.916999 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.916962 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj"] Jul 22 11:54:20.917419 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.917300 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:20.917419 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.917315 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:20.917419 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.917403 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:20.917419 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.917417 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:20.921811 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.921794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:20.924095 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.924076 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 11:54:20.924212 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.924080 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m4wzf\"" Jul 22 11:54:20.928010 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.927984 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj"] Jul 22 11:54:20.975778 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:20.975741 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-beed8e14-4swvj\" (UID: \"0e3cfc1a-6196-4837-b231-61af09c96214\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:21.076620 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.076552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-beed8e14-4swvj\" (UID: \"0e3cfc1a-6196-4837-b231-61af09c96214\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:21.077185 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.077165 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-beed8e14-4swvj\" (UID: \"0e3cfc1a-6196-4837-b231-61af09c96214\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:21.233735 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.233645 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:21.384924 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.384843 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj"] Jul 22 11:54:21.387366 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:54:21.387331 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3cfc1a_6196_4837_b231_61af09c96214.slice/crio-58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c WatchSource:0}: Error finding container 58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c: Status 404 returned error can't find the container with id 58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c Jul 22 11:54:21.887190 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.887160 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" event={"ID":"0e3cfc1a-6196-4837-b231-61af09c96214","Type":"ContainerStarted","Data":"6f9491ddca3afbe48db7aa3d4f4db609386a2fd8a6fad6c6e639570f82291303"} Jul 22 11:54:21.887360 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.887209 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" event={"ID":"0e3cfc1a-6196-4837-b231-61af09c96214","Type":"ContainerStarted","Data":"58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c"} Jul 22 11:54:21.911565 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:21.911516 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" podStartSLOduration=1.9115012789999999 podStartE2EDuration="1.911501279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:54:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:21.909825424 +0000 UTC m=+1830.417726311" watchObservedRunningTime="2026-07-22 11:54:21.911501279 +0000 UTC m=+1830.419402212" Jul 22 11:54:22.893141 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:22.893103 2577 generic.go:356] "Generic (PLEG): container finished" podID="0e3cfc1a-6196-4837-b231-61af09c96214" containerID="6f9491ddca3afbe48db7aa3d4f4db609386a2fd8a6fad6c6e639570f82291303" exitCode=0 Jul 22 11:54:22.893141 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:22.893147 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" event={"ID":"0e3cfc1a-6196-4837-b231-61af09c96214","Type":"ContainerDied","Data":"6f9491ddca3afbe48db7aa3d4f4db609386a2fd8a6fad6c6e639570f82291303"} Jul 22 11:54:24.028692 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.028668 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:24.102387 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.102356 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca\") pod \"0e3cfc1a-6196-4837-b231-61af09c96214\" (UID: \"0e3cfc1a-6196-4837-b231-61af09c96214\") " Jul 22 11:54:24.102727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.102706 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0e3cfc1a-6196-4837-b231-61af09c96214" (UID: "0e3cfc1a-6196-4837-b231-61af09c96214"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:54:24.203827 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.203742 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e3cfc1a-6196-4837-b231-61af09c96214-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:54:24.904722 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.904688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" event={"ID":"0e3cfc1a-6196-4837-b231-61af09c96214","Type":"ContainerDied","Data":"58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c"} Jul 22 11:54:24.904722 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.904721 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj" Jul 22 11:54:24.904940 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:24.904736 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="58e5cf9a68b684e3f0c7b0fd7d43feaa08f84fd2dd2b0580a2cf794d37172b6c" Jul 22 11:54:36.019556 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:36.019518 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj"] Jul 22 11:54:36.028480 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:36.028450 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-beed8e14-4swvj"] Jul 22 11:54:36.067295 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:36.067259 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e3cfc1a-6196-4837-b231-61af09c96214" path="/var/lib/kubelet/pods/0e3cfc1a-6196-4837-b231-61af09c96214/volumes" Jul 22 11:54:38.324712 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.324678 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:54:38.325099 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.324996 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:38.325099 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.325007 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="408fca5d-71eb-4706-b108-5c8326d9588f" containerName="istio-proxy" Jul 22 11:54:38.325099 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.325029 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e3cfc1a-6196-4837-b231-61af09c96214" containerName="revoke-keys" Jul 22 11:54:38.325099 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.325034 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e3cfc1a-6196-4837-b231-61af09c96214" containerName="revoke-keys" Jul 22 11:54:38.325099 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.325092 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e3cfc1a-6196-4837-b231-61af09c96214" containerName="revoke-keys" Jul 22 11:54:38.328360 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.328329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.330866 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.330835 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9906803d-openshift-default-dockercfg-65hz7\"" Jul 22 11:54:38.331024 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.330923 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:54:38.360276 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.360238 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:54:38.432167 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432125 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432167 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432164 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432380 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432200 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95cf4\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432380 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432380 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432380 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432353 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432380 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432378 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432679 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432407 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.432679 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.432473 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533140 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533095 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95cf4\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533140 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533145 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533178 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533240 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533317 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533438 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533722 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533754 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.533821 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533805 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.534003 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.533983 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.534230 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.534152 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.535847 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.535823 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.536045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.536030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.544710 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.544687 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.544814 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.544756 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95cf4\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4\") pod \"e2e-derive-9906803d-openshift-default-6559575455-x8vf5\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.639441 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.639399 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:38.790064 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.790039 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:54:38.792152 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:54:38.792121 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf22f3f0_908e_402b_b9c1_c73e4dfbc6e8.slice/crio-7907f324a5e60e0e4559100cd8d105169e922711e63dc268c4106f9474cfb7b7 WatchSource:0}: Error finding container 7907f324a5e60e0e4559100cd8d105169e922711e63dc268c4106f9474cfb7b7: Status 404 returned error can't find the container with id 7907f324a5e60e0e4559100cd8d105169e922711e63dc268c4106f9474cfb7b7 Jul 22 11:54:38.794169 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.794132 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:38.794243 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.794213 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:38.794279 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.794248 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:38.976727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.976621 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerStarted","Data":"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739"} Jul 22 11:54:38.976727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:38.976693 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerStarted","Data":"7907f324a5e60e0e4559100cd8d105169e922711e63dc268c4106f9474cfb7b7"} Jul 22 11:54:39.004441 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:39.004386 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podStartSLOduration=1.004368822 podStartE2EDuration="1.004368822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:54:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:54:39.003131942 +0000 UTC m=+1847.511032829" watchObservedRunningTime="2026-07-22 11:54:39.004368822 +0000 UTC m=+1847.512269709" Jul 22 11:54:39.640388 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:39.640354 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:39.645669 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:39.645643 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:39.982159 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:39.982076 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:39.983068 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:39.983047 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:52.046901 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.046817 2577 generic.go:356] "Generic (PLEG): container finished" podID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerID="42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739" exitCode=0 Jul 22 11:54:52.047509 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.046892 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerDied","Data":"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739"} Jul 22 11:54:52.047509 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.047281 2577 scope.go:117] "RemoveContainer" containerID="42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739" Jul 22 11:54:52.048230 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.048202 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:52.048278 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.048270 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:52.048320 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:52.048299 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:54:53.054452 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:53.054417 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerStarted","Data":"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614"} Jul 22 11:54:53.640150 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:53.640115 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:53.641197 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:53.641168 2577 patch_prober.go:28] interesting pod/e2e-derive-9906803d-openshift-default-6559575455-x8vf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 11:54:53.641328 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:53.641227 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 11:54:54.639986 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:54.639947 2577 patch_prober.go:28] interesting pod/e2e-derive-9906803d-openshift-default-6559575455-x8vf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 11:54:54.640389 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:54.640006 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 11:54:54.982703 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:54.982576 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:55.639974 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:55.639932 2577 patch_prober.go:28] interesting pod/e2e-derive-9906803d-openshift-default-6559575455-x8vf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 11:54:55.640168 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:55.639994 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 11:54:56.640522 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:56.640484 2577 patch_prober.go:28] interesting pod/e2e-derive-9906803d-openshift-default-6559575455-x8vf5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 22 11:54:56.640949 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:56.640544 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 22 11:54:57.643573 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:57.643542 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:54:57.644508 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:54:57.644486 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:55:20.614612 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:20.614569 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:55:20.615160 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:20.614851 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" containerID="cri-o://94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614" gracePeriod=30 Jul 22 11:55:25.880951 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.880865 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:55:25.978695 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978649 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978695 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978689 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978714 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978745 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978793 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978847 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978875 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95cf4\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978899 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.978952 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978939 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:55:25.979325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.978954 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token\") pod \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\" (UID: \"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8\") " Jul 22 11:55:25.979325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979099 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:55:25.979325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979292 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:25.979325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979310 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:25.979325 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979312 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:55:25.979585 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979569 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:55:25.979715 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.979696 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:55:25.981403 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.981360 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:55:25.981568 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.981549 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:55:25.981721 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.981699 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 11:55:25.981792 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:25.981731 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4" (OuterVolumeSpecName: "kube-api-access-95cf4") pod "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" (UID: "df22f3f0-908e-402b-b9c1-c73e4dfbc6e8"). InnerVolumeSpecName "kube-api-access-95cf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:55:26.079719 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079697 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079719 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079718 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079727 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079735 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079745 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95cf4\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-kube-api-access-95cf4\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079753 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.079872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.079761 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:55:26.222364 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.222275 2577 generic.go:356] "Generic (PLEG): container finished" podID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerID="94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614" exitCode=0 Jul 22 11:55:26.222364 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.222341 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" Jul 22 11:55:26.222565 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.222362 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerDied","Data":"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614"} Jul 22 11:55:26.222565 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.222417 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5" event={"ID":"df22f3f0-908e-402b-b9c1-c73e4dfbc6e8","Type":"ContainerDied","Data":"7907f324a5e60e0e4559100cd8d105169e922711e63dc268c4106f9474cfb7b7"} Jul 22 11:55:26.222565 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.222447 2577 scope.go:117] "RemoveContainer" containerID="94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614" Jul 22 11:55:26.233116 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.233076 2577 scope.go:117] "RemoveContainer" containerID="42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739" Jul 22 11:55:26.245543 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.245507 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:55:26.245870 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.245847 2577 scope.go:117] "RemoveContainer" containerID="94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614" Jul 22 11:55:26.246317 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:55:26.246287 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614\": container with ID starting with 94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614 not found: ID does not exist" containerID="94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614" Jul 22 11:55:26.246424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.246327 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614"} err="failed to get container status \"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614\": rpc error: code = NotFound desc = could not find container \"94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614\": container with ID starting with 94f4c1acb6a9d4a7c8daec1170b7f3c810f1d51365287f2e234eca96dafd6614 not found: ID does not exist" Jul 22 11:55:26.246424 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.246351 2577 scope.go:117] "RemoveContainer" containerID="42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739" Jul 22 11:55:26.246645 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:55:26.246611 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739\": container with ID starting with 42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739 not found: ID does not exist" containerID="42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739" Jul 22 11:55:26.246774 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.246642 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739"} err="failed to get container status \"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739\": rpc error: code = NotFound desc = could not find container \"42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739\": container with ID starting with 42304cfbd3f162bc93cd97f7565f3b75c8fc657c1cc6813880567f17e973e739 not found: ID does not exist" Jul 22 11:55:26.254543 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:26.254508 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9906803d-openshift-default-6559575455-x8vf5"] Jul 22 11:55:28.072119 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:55:28.068244 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" path="/var/lib/kubelet/pods/df22f3f0-908e-402b-b9c1-c73e4dfbc6e8/volumes" Jul 22 11:57:03.600373 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600282 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:03.600832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600681 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:57:03.600832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600695 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:57:03.600832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600705 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:57:03.600832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600711 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:57:03.600832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.600782 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:57:03.603543 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.603520 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.605810 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.605779 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8709147b-openshift-default-dockercfg-7ws6g\"" Jul 22 11:57:03.605942 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.605813 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:57:03.616554 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.616528 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:03.713518 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713577 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713627 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713727 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7ltn\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713923 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713764 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713923 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713809 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713923 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713847 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.713923 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.713870 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815208 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815234 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815284 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815312 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815351 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815408 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815429 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815470 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7ltn\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.815911 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815864 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.816045 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.815925 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.816113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.816038 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.816694 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.816668 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.816839 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.816816 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.818235 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.818200 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.818346 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.818287 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.825828 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.825792 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.825968 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.825878 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7ltn\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn\") pod \"e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:03.914357 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:03.914321 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:04.092429 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.092390 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:04.095836 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:57:04.095803 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590f221c_e2ae_4693_aa33_bdcd753d77f8.slice/crio-1ab5032be055f5055c8c549152bcc7d76a27989b9675880c7012939c10497448 WatchSource:0}: Error finding container 1ab5032be055f5055c8c549152bcc7d76a27989b9675880c7012939c10497448: Status 404 returned error can't find the container with id 1ab5032be055f5055c8c549152bcc7d76a27989b9675880c7012939c10497448 Jul 22 11:57:04.098274 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.098224 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:04.098397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.098315 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:04.098397 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.098358 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:04.749425 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.749386 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerStarted","Data":"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96"} Jul 22 11:57:04.749892 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.749449 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerStarted","Data":"1ab5032be055f5055c8c549152bcc7d76a27989b9675880c7012939c10497448"} Jul 22 11:57:04.776540 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.776480 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podStartSLOduration=1.776464292 podStartE2EDuration="1.776464292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:57:04.774771936 +0000 UTC m=+1993.282672822" watchObservedRunningTime="2026-07-22 11:57:04.776464292 +0000 UTC m=+1993.284365179" Jul 22 11:57:04.915280 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:04.915246 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:05.919774 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:05.919741 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:06.759694 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:06.759666 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:06.760734 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:06.760713 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:21.760931 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.760891 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:21.761363 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.760950 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:21.845123 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.845090 2577 generic.go:356] "Generic (PLEG): container finished" podID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerID="348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96" exitCode=0 Jul 22 11:57:21.845317 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.845169 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerDied","Data":"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96"} Jul 22 11:57:21.845587 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.845573 2577 scope.go:117] "RemoveContainer" containerID="348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96" Jul 22 11:57:21.846583 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.846556 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:21.846657 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.846640 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:21.846693 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:21.846676 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:57:22.852213 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:22.852179 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerStarted","Data":"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c"} Jul 22 11:57:22.915388 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:22.915347 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:22.915621 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:22.915573 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:22.915675 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:22.915642 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:23.915153 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:23.915114 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:23.915573 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:23.915176 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:24.915486 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:24.915444 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:24.915915 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:24.915521 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:25.915300 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:25.915263 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:25.915484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:25.915321 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:26.915749 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:26.915715 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 22 11:57:26.916140 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:26.915785 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 22 11:57:27.919080 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:27.919051 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:27.919461 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:27.919319 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:27.919983 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:27.919964 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:41.439562 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:41.439527 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:41.440061 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:41.439787 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" containerID="cri-o://80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c" gracePeriod=30 Jul 22 11:57:42.921353 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:42.921314 2577 patch_prober.go:28] interesting pod/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 11:57:42.921778 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:42.921372 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 11:57:46.691211 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.691176 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:46.804501 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804401 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804501 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804440 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804501 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804458 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804501 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804501 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7ltn\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804525 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804548 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804718 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804730 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804780 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.804889 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.804859 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs\") pod \"590f221c-e2ae-4693-aa33-bdcd753d77f8\" (UID: \"590f221c-e2ae-4693-aa33-bdcd753d77f8\") " Jul 22 11:57:46.805210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805023 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:57:46.805210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805175 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/590f221c-e2ae-4693-aa33-bdcd753d77f8-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.805210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805194 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.805210 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805162 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:46.805877 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805212 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:46.805963 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.805937 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:46.807028 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.807000 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 11:57:46.807139 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.807053 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 11:57:46.807139 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.807068 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:46.807217 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.807156 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn" (OuterVolumeSpecName: "kube-api-access-c7ltn") pod "590f221c-e2ae-4693-aa33-bdcd753d77f8" (UID: "590f221c-e2ae-4693-aa33-bdcd753d77f8"). InnerVolumeSpecName "kube-api-access-c7ltn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 11:57:46.905819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905771 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.905819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905810 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.905819 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905822 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.906078 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905833 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7ltn\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-kube-api-access-c7ltn\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.906078 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905846 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.906078 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905857 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.906078 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.905869 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/590f221c-e2ae-4693-aa33-bdcd753d77f8-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:57:46.980872 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.980834 2577 generic.go:356] "Generic (PLEG): container finished" podID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerID="80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c" exitCode=0 Jul 22 11:57:46.981082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.980883 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" Jul 22 11:57:46.981082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.980897 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerDied","Data":"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c"} Jul 22 11:57:46.981082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.980951 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d" event={"ID":"590f221c-e2ae-4693-aa33-bdcd753d77f8","Type":"ContainerDied","Data":"1ab5032be055f5055c8c549152bcc7d76a27989b9675880c7012939c10497448"} Jul 22 11:57:46.981082 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.980982 2577 scope.go:117] "RemoveContainer" containerID="80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c" Jul 22 11:57:46.990816 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.990796 2577 scope.go:117] "RemoveContainer" containerID="348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96" Jul 22 11:57:46.999055 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.999031 2577 scope.go:117] "RemoveContainer" containerID="80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c" Jul 22 11:57:46.999339 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:57:46.999320 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c\": container with ID starting with 80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c not found: ID does not exist" containerID="80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c" Jul 22 11:57:46.999388 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.999348 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c"} err="failed to get container status \"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c\": rpc error: code = NotFound desc = could not find container \"80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c\": container with ID starting with 80f372fcf11435a0f1cc95345066e6795edfeaeecbe65e6804f7e45ab9abbe3c not found: ID does not exist" Jul 22 11:57:46.999388 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.999365 2577 scope.go:117] "RemoveContainer" containerID="348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96" Jul 22 11:57:46.999575 ip-10-0-135-12 kubenswrapper[2577]: E0722 11:57:46.999559 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96\": container with ID starting with 348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96 not found: ID does not exist" containerID="348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96" Jul 22 11:57:46.999636 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:46.999580 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96"} err="failed to get container status \"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96\": rpc error: code = NotFound desc = could not find container \"348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96\": container with ID starting with 348d03a3f1c58cc3e2ef9bb69f240a2b1ccccda9e83a4e4b084c90ffda49cd96 not found: ID does not exist" Jul 22 11:57:47.008747 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:47.008719 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:47.016077 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:47.016052 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8709147b-openshift-default-86b5d8885f-69r2d"] Jul 22 11:57:48.067350 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:57:48.067306 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" path="/var/lib/kubelet/pods/590f221c-e2ae-4693-aa33-bdcd753d77f8/volumes" Jul 22 11:58:02.358496 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.358458 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 11:58:02.358979 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.358964 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:02.359025 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.358982 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:02.359078 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.359066 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:02.359113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.359086 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:02.359113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.359099 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="df22f3f0-908e-402b-b9c1-c73e4dfbc6e8" containerName="istio-proxy" Jul 22 11:58:02.362056 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.362038 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.364569 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.364545 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-98b5a8b0-openshift-default-dockercfg-bfc4z\"" Jul 22 11:58:02.364699 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.364620 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 11:58:02.381852 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.381819 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 11:58:02.546034 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.545996 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546034 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546032 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546067 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546217 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546253 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546314 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546305 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pqp\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546503 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.546503 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.546354 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.646897 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646811 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pqp\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.646897 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.646897 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646880 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647198 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646923 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647198 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646941 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647198 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.646983 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647198 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647156 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647421 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647221 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647421 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647262 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647421 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647266 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647577 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647653 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647616 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647885 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647656 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.647885 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.647689 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.649484 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.649458 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.649761 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.649744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.656969 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.656945 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.657206 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.657189 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pqp\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp\") pod \"e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.671895 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.671869 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:02.822579 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.822550 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 11:58:02.824402 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:58:02.824370 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f443b29_be76_4bc5_b213_6b47e5c041b9.slice/crio-e7ed63e03b063b128bdc3e7803bb460e25b68734c9e3a42e357abfec7982865e WatchSource:0}: Error finding container e7ed63e03b063b128bdc3e7803bb460e25b68734c9e3a42e357abfec7982865e: Status 404 returned error can't find the container with id e7ed63e03b063b128bdc3e7803bb460e25b68734c9e3a42e357abfec7982865e Jul 22 11:58:02.826630 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.826578 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:02.826686 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.826666 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:02.826728 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:02.826694 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:03.066456 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:03.066360 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerStarted","Data":"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57"} Jul 22 11:58:03.066456 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:03.066422 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerStarted","Data":"e7ed63e03b063b128bdc3e7803bb460e25b68734c9e3a42e357abfec7982865e"} Jul 22 11:58:03.088964 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:03.088905 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podStartSLOduration=1.088888107 podStartE2EDuration="1.088888107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:58:03.086751686 +0000 UTC m=+2051.594652573" watchObservedRunningTime="2026-07-22 11:58:03.088888107 +0000 UTC m=+2051.596788992" Jul 22 11:58:03.672230 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:03.672195 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:03.676949 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:03.676922 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:04.072094 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:04.072009 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:04.072939 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:04.072920 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:18.145272 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.145237 2577 generic.go:356] "Generic (PLEG): container finished" podID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerID="f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57" exitCode=0 Jul 22 11:58:18.145733 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.145315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerDied","Data":"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57"} Jul 22 11:58:18.145783 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.145769 2577 scope.go:117] "RemoveContainer" containerID="f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57" Jul 22 11:58:18.146887 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.146852 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:18.146990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.146945 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:18.146990 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:18.146986 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 11:58:19.072191 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:19.072153 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:19.153086 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:19.153038 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerStarted","Data":"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f"} Jul 22 11:58:19.672495 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:19.672450 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:19.673810 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:19.673785 2577 patch_prober.go:28] interesting pod/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 11:58:19.673909 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:19.673834 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 11:58:20.672425 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:20.672381 2577 patch_prober.go:28] interesting pod/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 11:58:20.672937 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:20.672443 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 11:58:21.672278 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:21.672241 2577 patch_prober.go:28] interesting pod/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 11:58:21.672439 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:21.672298 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 11:58:22.672822 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:22.672780 2577 patch_prober.go:28] interesting pod/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body= Jul 22 11:58:22.673224 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:22.672839 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" Jul 22 11:58:23.676901 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:23.676864 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:23.677374 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:23.677137 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:23.677803 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:23.677784 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 11:58:52.111746 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:52.111716 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:58:52.118832 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:52.118812 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 11:58:53.355731 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.355699 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v"] Jul 22 11:58:53.356113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.356048 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:53.356113 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.356058 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="590f221c-e2ae-4693-aa33-bdcd753d77f8" containerName="istio-proxy" Jul 22 11:58:53.359408 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.359391 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:53.361629 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.361604 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m4wzf\"" Jul 22 11:58:53.361744 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.361643 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 11:58:53.369190 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.369165 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v"] Jul 22 11:58:53.485020 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.484964 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v\" (UID: \"168ca100-229a-47c4-b241-718800afcabf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:53.586318 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.586286 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v\" (UID: \"168ca100-229a-47c4-b241-718800afcabf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:53.586936 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.586917 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v\" (UID: \"168ca100-229a-47c4-b241-718800afcabf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:53.670955 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.670866 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:53.806119 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:53.806084 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v"] Jul 22 11:58:53.809326 ip-10-0-135-12 kubenswrapper[2577]: W0722 11:58:53.809294 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod168ca100_229a_47c4_b241_718800afcabf.slice/crio-6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd WatchSource:0}: Error finding container 6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd: Status 404 returned error can't find the container with id 6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd Jul 22 11:58:54.331650 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:54.331526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" event={"ID":"168ca100-229a-47c4-b241-718800afcabf","Type":"ContainerStarted","Data":"580a0d666f78bb44ffa389116f0691e57a2d4003287332ee930e0610c76d5d55"} Jul 22 11:58:54.331650 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:54.331588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" event={"ID":"168ca100-229a-47c4-b241-718800afcabf","Type":"ContainerStarted","Data":"6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd"} Jul 22 11:58:54.349755 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:54.349702 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" podStartSLOduration=1.349685219 podStartE2EDuration="1.349685219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 11:58:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 11:58:54.347154903 +0000 UTC m=+2102.855055789" watchObservedRunningTime="2026-07-22 11:58:54.349685219 +0000 UTC m=+2102.857586135" Jul 22 11:58:55.337996 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:55.337957 2577 generic.go:356] "Generic (PLEG): container finished" podID="168ca100-229a-47c4-b241-718800afcabf" containerID="580a0d666f78bb44ffa389116f0691e57a2d4003287332ee930e0610c76d5d55" exitCode=0 Jul 22 11:58:55.338436 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:55.338044 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" event={"ID":"168ca100-229a-47c4-b241-718800afcabf","Type":"ContainerDied","Data":"580a0d666f78bb44ffa389116f0691e57a2d4003287332ee930e0610c76d5d55"} Jul 22 11:58:56.472482 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:56.472458 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:56.515549 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:56.515516 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca\") pod \"168ca100-229a-47c4-b241-718800afcabf\" (UID: \"168ca100-229a-47c4-b241-718800afcabf\") " Jul 22 11:58:56.515878 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:56.515853 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "168ca100-229a-47c4-b241-718800afcabf" (UID: "168ca100-229a-47c4-b241-718800afcabf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 11:58:56.616062 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:56.616028 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/168ca100-229a-47c4-b241-718800afcabf-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 11:58:57.348899 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:57.348864 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" event={"ID":"168ca100-229a-47c4-b241-718800afcabf","Type":"ContainerDied","Data":"6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd"} Jul 22 11:58:57.348899 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:57.348900 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v" Jul 22 11:58:57.349130 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:58:57.348914 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6592a04bc09e0ede817e0df61e888d64cdfda517c232e1d69db6a66dc90e99cd" Jul 22 11:59:13.505437 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:59:13.505395 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v"] Jul 22 11:59:13.512711 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:59:13.512686 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-98b5a8b0-d9k8v"] Jul 22 11:59:14.073066 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:59:14.073022 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="168ca100-229a-47c4-b241-718800afcabf" path="/var/lib/kubelet/pods/168ca100-229a-47c4-b241-718800afcabf/volumes" Jul 22 11:59:52.193918 ip-10-0-135-12 kubenswrapper[2577]: I0722 11:59:52.193843 2577 scope.go:117] "RemoveContainer" containerID="a155411d968c5165686502155e3c908c09d37b7ce894c005dd5c7d1ce80a98ab" Jul 22 12:00:05.428777 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:05.428737 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 12:00:05.429235 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:05.428995 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" containerID="cri-o://83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f" gracePeriod=30 Jul 22 12:00:08.679204 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:08.679163 2577 patch_prober.go:28] interesting pod/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 12:00:08.679655 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:08.679229 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 12:00:10.688665 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.686829 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 12:00:10.729574 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.729545 2577 generic.go:356] "Generic (PLEG): container finished" podID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerID="83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f" exitCode=0 Jul 22 12:00:10.729824 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.729587 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerDied","Data":"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f"} Jul 22 12:00:10.729824 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.729630 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" Jul 22 12:00:10.729824 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.729668 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9" event={"ID":"8f443b29-be76-4bc5-b213-6b47e5c041b9","Type":"ContainerDied","Data":"e7ed63e03b063b128bdc3e7803bb460e25b68734c9e3a42e357abfec7982865e"} Jul 22 12:00:10.729824 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.729694 2577 scope.go:117] "RemoveContainer" containerID="83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f" Jul 22 12:00:10.739495 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.739472 2577 scope.go:117] "RemoveContainer" containerID="f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57" Jul 22 12:00:10.750570 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.750544 2577 scope.go:117] "RemoveContainer" containerID="83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f" Jul 22 12:00:10.750958 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:00:10.750922 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f\": container with ID starting with 83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f not found: ID does not exist" containerID="83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f" Jul 22 12:00:10.751079 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.750958 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f"} err="failed to get container status \"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f\": rpc error: code = NotFound desc = could not find container \"83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f\": container with ID starting with 83554b57acb15e87ff80f2ccd882b0a31856a26332e65db78cdd3ac11849247f not found: ID does not exist" Jul 22 12:00:10.751079 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.750982 2577 scope.go:117] "RemoveContainer" containerID="f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57" Jul 22 12:00:10.751240 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:00:10.751223 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57\": container with ID starting with f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57 not found: ID does not exist" containerID="f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57" Jul 22 12:00:10.751301 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.751248 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57"} err="failed to get container status \"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57\": rpc error: code = NotFound desc = could not find container \"f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57\": container with ID starting with f89d9e48463e6b2af63db183ed8e3358bb11cbd2c73cd47a95adf5dc1682dd57 not found: ID does not exist" Jul 22 12:00:10.820194 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820161 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820213 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820244 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820264 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820312 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820351 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820742 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820532 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:00:10.820742 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820546 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:00:10.820853 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820749 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:00:10.820853 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820797 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.820853 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820841 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5pqp\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.821008 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.820875 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert\") pod \"8f443b29-be76-4bc5-b213-6b47e5c041b9\" (UID: \"8f443b29-be76-4bc5-b213-6b47e5c041b9\") " Jul 22 12:00:10.821235 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.821169 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:00:10.821235 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.821175 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.821235 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.821221 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.821396 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.821242 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.821396 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.821255 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:00:10.822901 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.822866 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:00:10.822901 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.822876 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:00:10.823119 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.823097 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:00:10.823540 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.823515 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp" (OuterVolumeSpecName: "kube-api-access-r5pqp") pod "8f443b29-be76-4bc5-b213-6b47e5c041b9" (UID: "8f443b29-be76-4bc5-b213-6b47e5c041b9"). InnerVolumeSpecName "kube-api-access-r5pqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:00:10.921918 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921882 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.921918 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921913 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.921918 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921924 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.922194 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921938 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f443b29-be76-4bc5-b213-6b47e5c041b9-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.922194 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921950 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5pqp\" (UniqueName: \"kubernetes.io/projected/8f443b29-be76-4bc5-b213-6b47e5c041b9-kube-api-access-r5pqp\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:10.922194 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:10.921961 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f443b29-be76-4bc5-b213-6b47e5c041b9-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:00:11.077396 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:11.077349 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 12:00:11.088857 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:11.088822 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-98b5a8b0-openshift-default-7d86d9f655-tp8l9"] Jul 22 12:00:12.067516 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:12.067479 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" path="/var/lib/kubelet/pods/8f443b29-be76-4bc5-b213-6b47e5c041b9/volumes" Jul 22 12:00:52.214451 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:00:52.214422 2577 scope.go:117] "RemoveContainer" containerID="6f9491ddca3afbe48db7aa3d4f4db609386a2fd8a6fad6c6e639570f82291303" Jul 22 12:01:03.615146 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615109 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:01:03.615618 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615587 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615622 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615638 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615647 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615662 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="168ca100-229a-47c4-b241-718800afcabf" containerName="revoke-keys" Jul 22 12:01:03.615678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615671 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="168ca100-229a-47c4-b241-718800afcabf" containerName="revoke-keys" Jul 22 12:01:03.615858 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615757 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615858 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615769 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f443b29-be76-4bc5-b213-6b47e5c041b9" containerName="istio-proxy" Jul 22 12:01:03.615858 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.615777 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="168ca100-229a-47c4-b241-718800afcabf" containerName="revoke-keys" Jul 22 12:01:03.619002 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.618980 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.622335 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.622313 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4750a4-openshift-default-dockercfg-rmvm2\"" Jul 22 12:01:03.622650 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.622628 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 12:01:03.640257 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.640224 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:01:03.779103 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779054 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779312 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779116 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779312 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779312 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779243 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9cwp\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779312 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779304 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779549 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779357 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779549 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779397 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779549 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779474 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.779549 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.779536 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880617 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880496 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880617 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880548 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880617 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880613 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9cwp\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880767 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.880907 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.881295 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.880949 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.881406 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.881363 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.881539 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.881519 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.881842 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.881813 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.881973 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.881821 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.883287 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.883260 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.883704 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.883683 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.893463 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.893437 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.893623 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.893554 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9cwp\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp\") pod \"e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:03.929426 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:03.929381 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:04.096476 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:04.096444 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:01:04.097399 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:01:04.097370 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa086c44_ee9d_4fa0_80c4_fcca5f5f1321.slice/crio-e174ab033b4bbeb7418c775ecc9edc5717e54020adce7f59fff1c8d56daeea60 WatchSource:0}: Error finding container e174ab033b4bbeb7418c775ecc9edc5717e54020adce7f59fff1c8d56daeea60: Status 404 returned error can't find the container with id e174ab033b4bbeb7418c775ecc9edc5717e54020adce7f59fff1c8d56daeea60 Jul 22 12:01:04.099637 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:04.099586 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:04.099722 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:04.099670 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:04.099722 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:04.099701 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:05.007550 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:05.007507 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerStarted","Data":"adec06b40c0ce2a51e171ee62ed37e016bf3575c0b15c1f727e52752b40fb789"} Jul 22 12:01:05.007973 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:05.007572 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerStarted","Data":"e174ab033b4bbeb7418c775ecc9edc5717e54020adce7f59fff1c8d56daeea60"} Jul 22 12:01:05.033518 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:05.033458 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podStartSLOduration=2.03344205 podStartE2EDuration="2.03344205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:05.030711942 +0000 UTC m=+2233.538612828" watchObservedRunningTime="2026-07-22 12:01:05.03344205 +0000 UTC m=+2233.541342936" Jul 22 12:01:05.930036 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:05.929996 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:05.934620 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:05.934569 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:06.012743 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:06.012712 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:06.013825 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:06.013805 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:12.222128 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.222033 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:01:12.226208 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.226188 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.228388 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.228361 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-aa23a9-openshift-default-dockercfg-4bd6h\"" Jul 22 12:01:12.240175 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.240146 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:01:12.251922 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.251895 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252040 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.251939 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252040 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.251981 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252040 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252005 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252165 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252089 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252207 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252155 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252250 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252206 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5nsc\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252302 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.252358 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.252303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353063 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353023 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353063 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353322 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353110 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353322 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353322 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353293 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353492 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353492 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5nsc\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353492 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353432 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353492 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353456 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353711 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353491 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353711 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353752 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.353869 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.353849 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.354082 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.354059 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.355856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.355830 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.356200 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.356180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.364334 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.364312 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5nsc\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.364334 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.364326 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token\") pod \"e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.537122 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.537003 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:12.692056 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.692021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:01:12.696872 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:01:12.696829 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddcc74da_1e6a_4220_b3f3_d1eb8a295567.slice/crio-d831acc2eaeba4a084d90f46490f5a399d39d8f24a4b4436ba7301c5a7f36362 WatchSource:0}: Error finding container d831acc2eaeba4a084d90f46490f5a399d39d8f24a4b4436ba7301c5a7f36362: Status 404 returned error can't find the container with id d831acc2eaeba4a084d90f46490f5a399d39d8f24a4b4436ba7301c5a7f36362 Jul 22 12:01:12.699060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.699027 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:12.699146 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.699094 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:12.699146 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:12.699122 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:13.050570 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:13.050526 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerStarted","Data":"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f"} Jul 22 12:01:13.050871 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:13.050608 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerStarted","Data":"d831acc2eaeba4a084d90f46490f5a399d39d8f24a4b4436ba7301c5a7f36362"} Jul 22 12:01:13.076422 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:13.076366 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podStartSLOduration=1.076345826 podStartE2EDuration="1.076345826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:01:13.075305014 +0000 UTC m=+2241.583205899" watchObservedRunningTime="2026-07-22 12:01:13.076345826 +0000 UTC m=+2241.584246713" Jul 22 12:01:13.537500 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:13.537459 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:13.542249 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:13.542220 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:14.055220 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:14.055185 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:14.056152 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:14.056133 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:21.014076 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.014040 2577 patch_prober.go:28] interesting pod/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 12:01:21.014505 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.014108 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 12:01:21.096222 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.096186 2577 generic.go:356] "Generic (PLEG): container finished" podID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerID="adec06b40c0ce2a51e171ee62ed37e016bf3575c0b15c1f727e52752b40fb789" exitCode=0 Jul 22 12:01:21.096404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.096263 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerDied","Data":"adec06b40c0ce2a51e171ee62ed37e016bf3575c0b15c1f727e52752b40fb789"} Jul 22 12:01:21.096695 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.096679 2577 scope.go:117] "RemoveContainer" containerID="adec06b40c0ce2a51e171ee62ed37e016bf3575c0b15c1f727e52752b40fb789" Jul 22 12:01:21.097720 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.097690 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:21.097818 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.097757 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:21.097818 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:21.097789 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:22.106376 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:22.106329 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerStarted","Data":"362ff8103b2b954c315275cb57d7e9bba405e439676ccba4a3d2205d8e1c7436"} Jul 22 12:01:22.930542 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:22.930495 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:22.930844 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:22.930819 2577 patch_prober.go:28] interesting pod/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 12:01:22.930927 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:22.930879 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 12:01:23.930388 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:23.930348 2577 patch_prober.go:28] interesting pod/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 12:01:23.930805 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:23.930418 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 12:01:24.930882 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:24.930836 2577 patch_prober.go:28] interesting pod/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 12:01:24.931290 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:24.930923 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 12:01:25.930195 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:25.930155 2577 patch_prober.go:28] interesting pod/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body= Jul 22 12:01:25.930408 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:25.930222 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" Jul 22 12:01:26.934912 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:26.934878 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:26.935400 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:26.935287 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:26.936218 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:26.936194 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:01:28.961729 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:01:28.961700 2577 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddcc74da_1e6a_4220_b3f3_d1eb8a295567.slice/crio-a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f.scope\": RecentStats: unable to find data in memory cache]" Jul 22 12:01:29.056639 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.056577 2577 patch_prober.go:28] interesting pod/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 12:01:29.056798 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.056669 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 12:01:29.154509 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.154413 2577 generic.go:356] "Generic (PLEG): container finished" podID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerID="a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f" exitCode=0 Jul 22 12:01:29.154509 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.154494 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerDied","Data":"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f"} Jul 22 12:01:29.154933 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.154919 2577 scope.go:117] "RemoveContainer" containerID="a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f" Jul 22 12:01:29.155894 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.155865 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:29.155996 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.155949 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:29.156060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:29.155997 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:01:30.161314 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:30.161278 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerStarted","Data":"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb"} Jul 22 12:01:30.537680 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:30.537550 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:30.537837 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:30.537768 2577 patch_prober.go:28] interesting pod/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 12:01:30.537837 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:30.537815 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 12:01:31.538009 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:31.537965 2577 patch_prober.go:28] interesting pod/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 12:01:31.538507 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:31.538033 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 12:01:32.537835 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:32.537795 2577 patch_prober.go:28] interesting pod/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 12:01:32.538029 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:32.537854 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 12:01:33.538513 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:33.538467 2577 patch_prober.go:28] interesting pod/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body= Jul 22 12:01:33.538982 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:33.538551 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" Jul 22 12:01:34.542216 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:34.542183 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:34.542703 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:34.542446 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:01:34.543176 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:01:34.543160 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:03:03.502094 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:03.502006 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:03:03.502616 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:03.502286 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" containerID="cri-o://362ff8103b2b954c315275cb57d7e9bba405e439676ccba4a3d2205d8e1c7436" gracePeriod=30 Jul 22 12:03:08.661582 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.661548 2577 generic.go:356] "Generic (PLEG): container finished" podID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerID="362ff8103b2b954c315275cb57d7e9bba405e439676ccba4a3d2205d8e1c7436" exitCode=0 Jul 22 12:03:08.662096 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.661627 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerDied","Data":"362ff8103b2b954c315275cb57d7e9bba405e439676ccba4a3d2205d8e1c7436"} Jul 22 12:03:08.662096 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.661689 2577 scope.go:117] "RemoveContainer" containerID="adec06b40c0ce2a51e171ee62ed37e016bf3575c0b15c1f727e52752b40fb789" Jul 22 12:03:08.758148 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.758126 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:03:08.814417 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814339 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814417 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814372 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814417 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814411 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814437 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814477 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814509 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814546 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9cwp\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814573 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.814799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814623 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket\") pod \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\" (UID: \"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321\") " Jul 22 12:03:08.815115 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814831 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:08.815115 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814938 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:03:08.815115 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.814954 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.815115 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.815053 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:08.815115 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.815069 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:08.816110 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.816081 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data" (OuterVolumeSpecName: "istio-data") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:08.816829 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.816806 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp" (OuterVolumeSpecName: "kube-api-access-r9cwp") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "kube-api-access-r9cwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:03:08.816928 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.816846 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:08.816928 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.816850 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token" (OuterVolumeSpecName: "istio-token") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:03:08.817022 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.816991 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" (UID: "fa086c44-ee9d-4fa0-80c4-fcca5f5f1321"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:03:08.915663 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915628 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915663 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915656 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915663 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915666 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915663 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915676 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915685 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915694 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9cwp\" (UniqueName: \"kubernetes.io/projected/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-kube-api-access-r9cwp\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915703 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:08.915934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:08.915712 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:09.668319 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:09.668290 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" Jul 22 12:03:09.668789 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:09.668289 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc" event={"ID":"fa086c44-ee9d-4fa0-80c4-fcca5f5f1321","Type":"ContainerDied","Data":"e174ab033b4bbeb7418c775ecc9edc5717e54020adce7f59fff1c8d56daeea60"} Jul 22 12:03:09.668789 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:09.668439 2577 scope.go:117] "RemoveContainer" containerID="362ff8103b2b954c315275cb57d7e9bba405e439676ccba4a3d2205d8e1c7436" Jul 22 12:03:09.692369 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:09.692324 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:03:09.694481 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:09.694458 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4750a4-openshift-default-6f4bf44f95-xsssc"] Jul 22 12:03:10.067779 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:10.067700 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" path="/var/lib/kubelet/pods/fa086c44-ee9d-4fa0-80c4-fcca5f5f1321/volumes" Jul 22 12:03:25.871478 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:25.871442 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:03:25.871961 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:25.871700 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" containerID="cri-o://e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb" gracePeriod=30 Jul 22 12:03:31.136619 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.136532 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:03:31.310796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310762 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.310796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310801 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310839 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5nsc\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310869 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310891 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310911 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310952 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.310991 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311041 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311022 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs\") pod \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\" (UID: \"ddcc74da-1e6a-4220-b3f3-d1eb8a295567\") " Jul 22 12:03:31.311386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311083 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:31.311386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311273 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:31.311386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311352 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:03:31.311557 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311397 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.311557 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311421 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.311557 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311394 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:31.311879 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.311760 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data" (OuterVolumeSpecName: "istio-data") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:31.313458 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.313431 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:03:31.313458 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.313435 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:03:31.313778 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.313748 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token" (OuterVolumeSpecName: "istio-token") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:03:31.313778 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.313764 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc" (OuterVolumeSpecName: "kube-api-access-l5nsc") pod "ddcc74da-1e6a-4220-b3f3-d1eb8a295567" (UID: "ddcc74da-1e6a-4220-b3f3-d1eb8a295567"). InnerVolumeSpecName "kube-api-access-l5nsc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.411972 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412001 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5nsc\" (UniqueName: \"kubernetes.io/projected/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-kube-api-access-l5nsc\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412015 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412024 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412045 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412055 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.412060 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.412064 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ddcc74da-1e6a-4220-b3f3-d1eb8a295567-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:31.786327 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.786234 2577 generic.go:356] "Generic (PLEG): container finished" podID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerID="e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb" exitCode=0 Jul 22 12:03:31.786327 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.786282 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" Jul 22 12:03:31.786327 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.786315 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerDied","Data":"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb"} Jul 22 12:03:31.786697 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.786371 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l" event={"ID":"ddcc74da-1e6a-4220-b3f3-d1eb8a295567","Type":"ContainerDied","Data":"d831acc2eaeba4a084d90f46490f5a399d39d8f24a4b4436ba7301c5a7f36362"} Jul 22 12:03:31.786697 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.786398 2577 scope.go:117] "RemoveContainer" containerID="e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb" Jul 22 12:03:31.796517 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.796500 2577 scope.go:117] "RemoveContainer" containerID="a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f" Jul 22 12:03:31.804778 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.804762 2577 scope.go:117] "RemoveContainer" containerID="e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb" Jul 22 12:03:31.805044 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:03:31.805007 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb\": container with ID starting with e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb not found: ID does not exist" containerID="e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb" Jul 22 12:03:31.805092 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.805056 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb"} err="failed to get container status \"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb\": rpc error: code = NotFound desc = could not find container \"e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb\": container with ID starting with e4123e3840917a58433310d148cda7df1058d083a34586f8afbc9be893ea00bb not found: ID does not exist" Jul 22 12:03:31.805092 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.805071 2577 scope.go:117] "RemoveContainer" containerID="a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f" Jul 22 12:03:31.805271 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:03:31.805256 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f\": container with ID starting with a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f not found: ID does not exist" containerID="a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f" Jul 22 12:03:31.805315 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.805276 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f"} err="failed to get container status \"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f\": rpc error: code = NotFound desc = could not find container \"a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f\": container with ID starting with a9933cee5bda11453eab7267c9cd918d34fbac264200faf257316ce88232549f not found: ID does not exist" Jul 22 12:03:31.819531 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.819502 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:03:31.833791 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:31.833760 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-aa23a9-openshift-default-6b4b6cbf9c-sh84l"] Jul 22 12:03:32.067621 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:32.067527 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" path="/var/lib/kubelet/pods/ddcc74da-1e6a-4220-b3f3-d1eb8a295567/volumes" Jul 22 12:03:35.908398 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908356 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:03:35.908796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908758 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:35.908796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908772 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:35.908796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908781 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:35.908796 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908788 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908803 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908808 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908817 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908822 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908896 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908903 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.908935 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.908908 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddcc74da-1e6a-4220-b3f3-d1eb8a295567" containerName="istio-proxy" Jul 22 12:03:35.913410 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.913392 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.915743 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.915721 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-f39058-openshift-default-dockercfg-sp9wj\"" Jul 22 12:03:35.915951 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.915937 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 12:03:35.927440 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.927418 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:03:35.952254 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952229 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952268 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952288 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdt5w\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952387 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952535 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952535 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952450 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952535 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952488 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952688 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952533 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:35.952688 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:35.952642 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053638 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053604 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053697 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053718 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdt5w\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.053839 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053817 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054170 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.053975 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054170 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054170 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054034 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054170 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054107 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054358 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054358 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054316 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.054697 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.054634 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.056136 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.056113 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.056347 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.056329 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.063728 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.063704 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdt5w\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.068316 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.068295 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token\") pod \"e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.223971 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.223874 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:36.367497 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.367468 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:03:36.369477 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:03:36.369449 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ebf592_d02e_4279_bb3a_887aeb001a23.slice/crio-7b43fdeff9c131156e07365aa885616a1e6e82e4c288cc11508f4b44674fdb1e WatchSource:0}: Error finding container 7b43fdeff9c131156e07365aa885616a1e6e82e4c288cc11508f4b44674fdb1e: Status 404 returned error can't find the container with id 7b43fdeff9c131156e07365aa885616a1e6e82e4c288cc11508f4b44674fdb1e Jul 22 12:03:36.371495 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.371463 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:36.371588 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.371531 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:36.371588 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.371559 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:36.813367 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.813279 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerStarted","Data":"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974"} Jul 22 12:03:36.813367 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.813337 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerStarted","Data":"7b43fdeff9c131156e07365aa885616a1e6e82e4c288cc11508f4b44674fdb1e"} Jul 22 12:03:36.835697 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:36.835646 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podStartSLOduration=1.835629721 podStartE2EDuration="1.835629721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:03:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:03:36.834138571 +0000 UTC m=+2385.342039476" watchObservedRunningTime="2026-07-22 12:03:36.835629721 +0000 UTC m=+2385.343530601" Jul 22 12:03:37.224656 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:37.224622 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:37.229742 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:37.229717 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:37.818288 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:37.818250 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:37.819227 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:37.819208 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:52.147916 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.147882 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 12:03:52.154699 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.154678 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 12:03:52.231200 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.231164 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht"] Jul 22 12:03:52.231631 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.231617 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa086c44-ee9d-4fa0-80c4-fcca5f5f1321" containerName="istio-proxy" Jul 22 12:03:52.235866 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.235849 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:52.238347 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.238313 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 12:03:52.238347 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.238317 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m4wzf\"" Jul 22 12:03:52.244759 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.244734 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht"] Jul 22 12:03:52.314181 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.314142 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2dbb42-trxht\" (UID: \"fdf068df-07e3-41ee-add9-962c4c9f96a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:52.414742 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.414657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2dbb42-trxht\" (UID: \"fdf068df-07e3-41ee-add9-962c4c9f96a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:52.415277 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.415255 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2dbb42-trxht\" (UID: \"fdf068df-07e3-41ee-add9-962c4c9f96a7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:52.549179 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.549138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:52.694349 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.694322 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht"] Jul 22 12:03:52.696389 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:03:52.696362 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf068df_07e3_41ee_add9_962c4c9f96a7.slice/crio-0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61 WatchSource:0}: Error finding container 0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61: Status 404 returned error can't find the container with id 0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61 Jul 22 12:03:52.901918 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:52.901884 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" event={"ID":"fdf068df-07e3-41ee-add9-962c4c9f96a7","Type":"ContainerStarted","Data":"0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61"} Jul 22 12:03:53.018291 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.018261 2577 patch_prober.go:28] interesting pod/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:39844->10.134.0.54:15021: read: connection reset by peer" start-of-body= Jul 22 12:03:53.018452 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.018346 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": read tcp 10.134.0.2:39844->10.134.0.54:15021: read: connection reset by peer" Jul 22 12:03:53.907877 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.907841 2577 generic.go:356] "Generic (PLEG): container finished" podID="fdf068df-07e3-41ee-add9-962c4c9f96a7" containerID="5c9ef8efc465ad9f2584a9027e008aae35af85883959953ae1415a9e26eed10f" exitCode=0 Jul 22 12:03:53.908328 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.907929 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" event={"ID":"fdf068df-07e3-41ee-add9-962c4c9f96a7","Type":"ContainerDied","Data":"5c9ef8efc465ad9f2584a9027e008aae35af85883959953ae1415a9e26eed10f"} Jul 22 12:03:53.909425 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.909403 2577 generic.go:356] "Generic (PLEG): container finished" podID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerID="dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974" exitCode=0 Jul 22 12:03:53.909507 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.909470 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerDied","Data":"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974"} Jul 22 12:03:53.909831 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.909817 2577 scope.go:117] "RemoveContainer" containerID="dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974" Jul 22 12:03:53.910751 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.910722 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:53.910848 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.910807 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:53.910916 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:53.910860 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:03:54.916537 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:54.916499 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerStarted","Data":"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953"} Jul 22 12:03:55.066364 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.066337 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:55.137214 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.137176 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca\") pod \"fdf068df-07e3-41ee-add9-962c4c9f96a7\" (UID: \"fdf068df-07e3-41ee-add9-962c4c9f96a7\") " Jul 22 12:03:55.137544 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.137518 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fdf068df-07e3-41ee-add9-962c4c9f96a7" (UID: "fdf068df-07e3-41ee-add9-962c4c9f96a7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:03:55.225075 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.224982 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:55.225274 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.225250 2577 patch_prober.go:28] interesting pod/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 22 12:03:55.225351 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.225310 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 22 12:03:55.237722 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.237695 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdf068df-07e3-41ee-add9-962c4c9f96a7-maas-api-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:03:55.923229 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.923201 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" Jul 22 12:03:55.923677 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.923241 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht" event={"ID":"fdf068df-07e3-41ee-add9-962c4c9f96a7","Type":"ContainerDied","Data":"0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61"} Jul 22 12:03:55.923677 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:55.923288 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0292ebb3fb27920c00e3ce3a90736cfb6389ce2b2a094935ee058140bb796e61" Jul 22 12:03:56.225134 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:56.225043 2577 patch_prober.go:28] interesting pod/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 22 12:03:56.225134 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:56.225113 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 22 12:03:57.224722 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:57.224688 2577 patch_prober.go:28] interesting pod/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 22 12:03:57.225142 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:57.224747 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 22 12:03:58.224954 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:58.224912 2577 patch_prober.go:28] interesting pod/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body= Jul 22 12:03:58.225369 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:58.224987 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" Jul 22 12:03:59.228214 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:59.228180 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:59.228729 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:59.228582 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:03:59.229391 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:03:59.229369 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:04:12.344972 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:12.344887 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht"] Jul 22 12:04:12.347919 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:12.347889 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2dbb42-trxht"] Jul 22 12:04:14.068391 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:14.068355 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdf068df-07e3-41ee-add9-962c4c9f96a7" path="/var/lib/kubelet/pods/fdf068df-07e3-41ee-add9-962c4c9f96a7/volumes" Jul 22 12:04:46.573386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:46.573349 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:04:46.573883 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:46.573632 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" containerID="cri-o://e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953" gracePeriod=30 Jul 22 12:04:51.819853 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.819828 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:04:51.937057 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937027 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937057 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937068 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937098 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937115 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937131 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937162 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937198 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937243 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdt5w\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937305 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937278 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert\") pod \"44ebf592-d02e-4279-bb3a-887aeb001a23\" (UID: \"44ebf592-d02e-4279-bb3a-887aeb001a23\") " Jul 22 12:04:51.937688 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937518 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:04:51.937688 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937530 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:04:51.937809 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.937785 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:04:51.938158 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.938133 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data" (OuterVolumeSpecName: "istio-data") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:04:51.938313 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.938284 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:04:51.939687 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.939657 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:04:51.939801 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.939722 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w" (OuterVolumeSpecName: "kube-api-access-kdt5w") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "kube-api-access-kdt5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:04:51.939926 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.939896 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token" (OuterVolumeSpecName: "istio-token") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:04:51.940029 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:51.939963 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44ebf592-d02e-4279-bb3a-887aeb001a23" (UID: "44ebf592-d02e-4279-bb3a-887aeb001a23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:04:52.039046 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039009 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039047 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039070 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdt5w\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-kube-api-access-kdt5w\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039082 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44ebf592-d02e-4279-bb3a-887aeb001a23-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039096 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039107 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039116 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039134 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.039246 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.039145 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44ebf592-d02e-4279-bb3a-887aeb001a23-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:04:52.206018 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.205922 2577 generic.go:356] "Generic (PLEG): container finished" podID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerID="e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953" exitCode=0 Jul 22 12:04:52.206018 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.205990 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" Jul 22 12:04:52.206219 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.206014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerDied","Data":"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953"} Jul 22 12:04:52.206219 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.206072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n" event={"ID":"44ebf592-d02e-4279-bb3a-887aeb001a23","Type":"ContainerDied","Data":"7b43fdeff9c131156e07365aa885616a1e6e82e4c288cc11508f4b44674fdb1e"} Jul 22 12:04:52.206219 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.206094 2577 scope.go:117] "RemoveContainer" containerID="e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953" Jul 22 12:04:52.215415 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.215395 2577 scope.go:117] "RemoveContainer" containerID="dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974" Jul 22 12:04:52.224226 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.224204 2577 scope.go:117] "RemoveContainer" containerID="e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953" Jul 22 12:04:52.224504 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:04:52.224484 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953\": container with ID starting with e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953 not found: ID does not exist" containerID="e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953" Jul 22 12:04:52.224569 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.224511 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953"} err="failed to get container status \"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953\": rpc error: code = NotFound desc = could not find container \"e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953\": container with ID starting with e67c7c4080d51136ae3ed587811849d032ce240462f8165b418c668777a1e953 not found: ID does not exist" Jul 22 12:04:52.224569 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.224527 2577 scope.go:117] "RemoveContainer" containerID="dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974" Jul 22 12:04:52.224801 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:04:52.224781 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974\": container with ID starting with dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974 not found: ID does not exist" containerID="dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974" Jul 22 12:04:52.224864 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.224805 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974"} err="failed to get container status \"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974\": rpc error: code = NotFound desc = could not find container \"dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974\": container with ID starting with dec063d63438e3beff6470a2dbd39cadfa3fefd72a3ed58d127bfc451b221974 not found: ID does not exist" Jul 22 12:04:52.225563 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.225539 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:04:52.229498 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:52.229473 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-f39058-openshift-default-f7dfb5475-66t9n"] Jul 22 12:04:54.067672 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:54.067636 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" path="/var/lib/kubelet/pods/44ebf592-d02e-4279-bb3a-887aeb001a23/volumes" Jul 22 12:04:55.816874 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.816837 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817158 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdf068df-07e3-41ee-add9-962c4c9f96a7" containerName="revoke-keys" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817170 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdf068df-07e3-41ee-add9-962c4c9f96a7" containerName="revoke-keys" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817184 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817190 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817206 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817211 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817266 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:55.817300 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.817280 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdf068df-07e3-41ee-add9-962c4c9f96a7" containerName="revoke-keys" Jul 22 12:04:55.820872 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.820848 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.823485 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.823463 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 12:04:55.823631 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.823463 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ae7383-openshift-default-dockercfg-ck269\"" Jul 22 12:04:55.831652 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.831625 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:04:55.973443 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973408 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973674 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973674 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973674 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973618 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973674 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973654 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973910 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973689 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pjbn\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973910 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973723 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973910 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973803 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:55.973910 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:55.973832 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.074737 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.074737 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.074737 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074724 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074805 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074840 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074888 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pjbn\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.074920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075288 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.075140 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075530 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.075504 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075530 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.075519 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075745 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.075588 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.075745 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.075728 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.077762 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.077744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.077985 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.077963 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.084386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.084361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.084663 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.084642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pjbn\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn\") pod \"e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.133933 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.133897 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:56.278934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.278898 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:04:56.282612 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:04:56.282420 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6634b8f0_b094_4902_9934_a8d1fa9ad5b9.slice/crio-53b36359add92ccf794fac252e5bf8db3d69faafc7a85fb36749bc36606ca200 WatchSource:0}: Error finding container 53b36359add92ccf794fac252e5bf8db3d69faafc7a85fb36749bc36606ca200: Status 404 returned error can't find the container with id 53b36359add92ccf794fac252e5bf8db3d69faafc7a85fb36749bc36606ca200 Jul 22 12:04:56.290268 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.290232 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:04:56.290398 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.290297 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:04:56.290398 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:56.290327 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:04:57.234197 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:57.234163 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerStarted","Data":"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47"} Jul 22 12:04:57.234625 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:57.234238 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerStarted","Data":"53b36359add92ccf794fac252e5bf8db3d69faafc7a85fb36749bc36606ca200"} Jul 22 12:04:57.256317 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:57.256270 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podStartSLOduration=2.256253739 podStartE2EDuration="2.256253739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:04:57.254157452 +0000 UTC m=+2465.762058341" watchObservedRunningTime="2026-07-22 12:04:57.256253739 +0000 UTC m=+2465.764154701" Jul 22 12:04:58.135081 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:58.135049 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:58.139835 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:58.139812 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:58.238932 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:58.238909 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:58.239865 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:58.239847 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:04:59.287404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.287373 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:04:59.287966 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.287947 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="44ebf592-d02e-4279-bb3a-887aeb001a23" containerName="istio-proxy" Jul 22 12:04:59.291039 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.291019 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.293490 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.293464 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-ae7383-serving-cert\"" Jul 22 12:04:59.293689 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.293479 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 12:04:59.293770 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.293505 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rrgrs\"" Jul 22 12:04:59.294070 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.294049 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 12:04:59.302386 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.302361 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:04:59.404980 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.404950 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.405199 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.405014 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8pd\" (UniqueName: \"kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.405199 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.405075 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.506109 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.506065 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.506280 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.506183 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.506280 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.506222 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8pd\" (UniqueName: \"kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.506430 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:04:59.506319 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-serving-cert: secret "maas-api-e2e-shared-b-ae7383-serving-cert" not found Jul 22 12:04:59.506430 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:04:59.506382 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls podName:a7cf5b8a-79dd-44a4-b5f1-f7738d98669a nodeName:}" failed. No retries permitted until 2026-07-22 12:05:00.006366444 +0000 UTC m=+2468.514267308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls") pod "maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" (UID: "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a") : secret "maas-api-e2e-shared-b-ae7383-serving-cert" not found Jul 22 12:04:59.506720 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.506701 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:04:59.515688 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:04:59.515663 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8pd\" (UniqueName: \"kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:00.010622 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:00.010557 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:00.013198 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:00.013171 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") pod \"maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:00.204321 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:00.204284 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:00.345794 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:00.345769 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:05:00.347782 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:05:00.347757 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cf5b8a_79dd_44a4_b5f1_f7738d98669a.slice/crio-0f9fdb2bb0aaa07bcf53534c7bc99a628c7bbed760233a38a3da64df59551878 WatchSource:0}: Error finding container 0f9fdb2bb0aaa07bcf53534c7bc99a628c7bbed760233a38a3da64df59551878: Status 404 returned error can't find the container with id 0f9fdb2bb0aaa07bcf53534c7bc99a628c7bbed760233a38a3da64df59551878 Jul 22 12:05:00.349625 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:00.349590 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 12:05:01.255800 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:01.255763 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" event={"ID":"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a","Type":"ContainerStarted","Data":"0f9fdb2bb0aaa07bcf53534c7bc99a628c7bbed760233a38a3da64df59551878"} Jul 22 12:05:04.273003 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:04.272894 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" event={"ID":"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a","Type":"ContainerStarted","Data":"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8"} Jul 22 12:05:04.273003 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:04.272965 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:04.291173 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:04.291123 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" podStartSLOduration=1.642175717 podStartE2EDuration="5.291108555s" totalImagesPullingTime="3.648932838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:05:04.288637515 +0000 UTC m=+2472.796538402" watchObservedRunningTime="2026-07-22 12:05:04.291108555 +0000 UTC m=+2472.799009440" Jul 22 12:05:10.285349 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:10.285314 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:05:13.240444 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.240355 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 12:05:13.240444 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.240418 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 12:05:13.333913 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.333878 2577 generic.go:356] "Generic (PLEG): container finished" podID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerID="dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47" exitCode=0 Jul 22 12:05:13.334078 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.333956 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerDied","Data":"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47"} Jul 22 12:05:13.334348 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.334327 2577 scope.go:117] "RemoveContainer" containerID="dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47" Jul 22 12:05:13.335298 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.335270 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:05:13.335356 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.335335 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:05:13.335396 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:13.335370 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 22 12:05:14.341725 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:14.341688 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerStarted","Data":"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5"} Jul 22 12:05:15.134284 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:15.134244 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:05:15.134495 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:15.134471 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 12:05:15.134554 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:15.134536 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 12:05:16.135312 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:16.135273 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 12:05:16.135742 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:16.135335 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 12:05:17.134470 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:17.134428 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 12:05:17.134716 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:17.134492 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 12:05:18.134996 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:18.134962 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Jul 22 12:05:18.135397 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:18.135017 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Jul 22 12:05:19.139070 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:19.139040 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:05:19.139456 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:19.139389 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:05:19.140141 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:19.140123 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:05:52.287019 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:05:52.286946 2577 scope.go:117] "RemoveContainer" containerID="580a0d666f78bb44ffa389116f0691e57a2d4003287332ee930e0610c76d5d55" Jul 22 12:08:52.189229 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:08:52.189202 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 12:08:52.193669 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:08:52.193637 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dhtvd_8a8bada0-b5c1-4f11-a27d-c86e5f1d82ae/ovn-acl-logging/0.log" Jul 22 12:10:44.946174 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:44.946140 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771877296"] Jul 22 12:10:44.949347 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:44.949329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:44.951435 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:44.951414 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-cqns9\"" Jul 22 12:10:44.955408 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:44.955382 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771877296"] Jul 22 12:10:44.977968 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:44.977938 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87cn7\" (UniqueName: \"kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7\") pod \"test-curl-7897-140477771877296\" (UID: \"cb33ab67-a934-4390-8e68-8492ffc8484d\") " pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:45.079005 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:45.078974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87cn7\" (UniqueName: \"kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7\") pod \"test-curl-7897-140477771877296\" (UID: \"cb33ab67-a934-4390-8e68-8492ffc8484d\") " pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:45.088231 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:45.088201 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87cn7\" (UniqueName: \"kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7\") pod \"test-curl-7897-140477771877296\" (UID: \"cb33ab67-a934-4390-8e68-8492ffc8484d\") " pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:45.260175 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:45.260082 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:45.393012 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:45.392985 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771877296"] Jul 22 12:10:45.395235 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:10:45.395210 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb33ab67_a934_4390_8e68_8492ffc8484d.slice/crio-d1c635a6e5bccf4d7161dbb74efe52d3de4ab52075240deca88686c1efe2f120 WatchSource:0}: Error finding container d1c635a6e5bccf4d7161dbb74efe52d3de4ab52075240deca88686c1efe2f120: Status 404 returned error can't find the container with id d1c635a6e5bccf4d7161dbb74efe52d3de4ab52075240deca88686c1efe2f120 Jul 22 12:10:45.396885 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:45.396865 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 12:10:46.066931 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:46.066853 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771877296" event={"ID":"cb33ab67-a934-4390-8e68-8492ffc8484d","Type":"ContainerStarted","Data":"d1c635a6e5bccf4d7161dbb74efe52d3de4ab52075240deca88686c1efe2f120"} Jul 22 12:10:47.072369 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:47.072336 2577 generic.go:356] "Generic (PLEG): container finished" podID="cb33ab67-a934-4390-8e68-8492ffc8484d" containerID="a5878365cb379a59b20d347e388cb5fedabc96c247c4b8a3865918c58815a3d9" exitCode=0 Jul 22 12:10:47.072787 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:47.072414 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771877296" event={"ID":"cb33ab67-a934-4390-8e68-8492ffc8484d","Type":"ContainerDied","Data":"a5878365cb379a59b20d347e388cb5fedabc96c247c4b8a3865918c58815a3d9"} Jul 22 12:10:48.210335 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.210316 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:48.243488 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.243450 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771877296_cb33ab67-a934-4390-8e68-8492ffc8484d/test-curl-7897-140477771877296/0.log" Jul 22 12:10:48.267869 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.267827 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771877296"] Jul 22 12:10:48.273119 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.273092 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771877296"] Jul 22 12:10:48.307372 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.307346 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87cn7\" (UniqueName: \"kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7\") pod \"cb33ab67-a934-4390-8e68-8492ffc8484d\" (UID: \"cb33ab67-a934-4390-8e68-8492ffc8484d\") " Jul 22 12:10:48.309889 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.309855 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7" (OuterVolumeSpecName: "kube-api-access-87cn7") pod "cb33ab67-a934-4390-8e68-8492ffc8484d" (UID: "cb33ab67-a934-4390-8e68-8492ffc8484d"). InnerVolumeSpecName "kube-api-access-87cn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:10:48.398472 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.398430 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771877808"] Jul 22 12:10:48.399445 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.399409 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb33ab67-a934-4390-8e68-8492ffc8484d" containerName="test-curl-7897-140477771877296" Jul 22 12:10:48.399645 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.399617 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb33ab67-a934-4390-8e68-8492ffc8484d" containerName="test-curl-7897-140477771877296" Jul 22 12:10:48.399760 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.399727 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb33ab67-a934-4390-8e68-8492ffc8484d" containerName="test-curl-7897-140477771877296" Jul 22 12:10:48.402854 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.402837 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:48.406675 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.406648 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771877808"] Jul 22 12:10:48.408278 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.408232 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87cn7\" (UniqueName: \"kubernetes.io/projected/cb33ab67-a934-4390-8e68-8492ffc8484d-kube-api-access-87cn7\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:10:48.509548 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.509507 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q74n\" (UniqueName: \"kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n\") pod \"test-curl-7897-140477771877808\" (UID: \"fb54953c-685a-435a-93b2-c3f259abf640\") " pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:48.610544 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.610506 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q74n\" (UniqueName: \"kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n\") pod \"test-curl-7897-140477771877808\" (UID: \"fb54953c-685a-435a-93b2-c3f259abf640\") " pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:48.619808 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.619773 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q74n\" (UniqueName: \"kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n\") pod \"test-curl-7897-140477771877808\" (UID: \"fb54953c-685a-435a-93b2-c3f259abf640\") " pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:48.715000 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.714893 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:48.851021 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:48.850997 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771877808"] Jul 22 12:10:48.852760 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:10:48.852733 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb54953c_685a_435a_93b2_c3f259abf640.slice/crio-bc5bb1519dc3d1a83981fc0e65026509a38213ad976e6938e8f3c0f3484f4bed WatchSource:0}: Error finding container bc5bb1519dc3d1a83981fc0e65026509a38213ad976e6938e8f3c0f3484f4bed: Status 404 returned error can't find the container with id bc5bb1519dc3d1a83981fc0e65026509a38213ad976e6938e8f3c0f3484f4bed Jul 22 12:10:49.082722 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:49.082636 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771877808" event={"ID":"fb54953c-685a-435a-93b2-c3f259abf640","Type":"ContainerStarted","Data":"bc5bb1519dc3d1a83981fc0e65026509a38213ad976e6938e8f3c0f3484f4bed"} Jul 22 12:10:49.083881 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:49.083861 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:49.083963 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:49.083879 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1c635a6e5bccf4d7161dbb74efe52d3de4ab52075240deca88686c1efe2f120" Jul 22 12:10:49.093650 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:10:49.093624 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7897-140477771877296\" is forbidden: User \"system:node:ip-10-0-135-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-12.ec2.internal' and this object" podUID="cb33ab67-a934-4390-8e68-8492ffc8484d" pod="opendatahub/test-curl-7897-140477771877296" Jul 22 12:10:50.079287 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:50.079248 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb33ab67-a934-4390-8e68-8492ffc8484d" path="/var/lib/kubelet/pods/cb33ab67-a934-4390-8e68-8492ffc8484d/volumes" Jul 22 12:10:50.090748 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:50.090712 2577 generic.go:356] "Generic (PLEG): container finished" podID="fb54953c-685a-435a-93b2-c3f259abf640" containerID="b3e82f6bcdae6eba3b4ba59c99c2557cd10d423c0727d0b7b546d4df17c6355a" exitCode=0 Jul 22 12:10:50.090938 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:50.090763 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771877808" event={"ID":"fb54953c-685a-435a-93b2-c3f259abf640","Type":"ContainerDied","Data":"b3e82f6bcdae6eba3b4ba59c99c2557cd10d423c0727d0b7b546d4df17c6355a"} Jul 22 12:10:51.219854 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.219828 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:51.242102 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.242078 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771877808_fb54953c-685a-435a-93b2-c3f259abf640/test-curl-7897-140477771877808/0.log" Jul 22 12:10:51.266862 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.266823 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771877808"] Jul 22 12:10:51.269806 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.269778 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771877808"] Jul 22 12:10:51.334498 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.334462 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q74n\" (UniqueName: \"kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n\") pod \"fb54953c-685a-435a-93b2-c3f259abf640\" (UID: \"fb54953c-685a-435a-93b2-c3f259abf640\") " Jul 22 12:10:51.336743 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.336710 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n" (OuterVolumeSpecName: "kube-api-access-7q74n") pod "fb54953c-685a-435a-93b2-c3f259abf640" (UID: "fb54953c-685a-435a-93b2-c3f259abf640"). InnerVolumeSpecName "kube-api-access-7q74n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:10:51.435714 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.435679 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q74n\" (UniqueName: \"kubernetes.io/projected/fb54953c-685a-435a-93b2-c3f259abf640-kube-api-access-7q74n\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:10:51.440277 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.440248 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:51.440655 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.440629 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb54953c-685a-435a-93b2-c3f259abf640" containerName="test-curl-7897-140477771877808" Jul 22 12:10:51.440655 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.440644 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb54953c-685a-435a-93b2-c3f259abf640" containerName="test-curl-7897-140477771877808" Jul 22 12:10:51.440750 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.440723 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb54953c-685a-435a-93b2-c3f259abf640" containerName="test-curl-7897-140477771877808" Jul 22 12:10:51.443762 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.443747 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:51.449101 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.449076 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:51.536665 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.536587 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jglzv\" (UniqueName: \"kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv\") pod \"test-curl-7897-140477771908128\" (UID: \"24a917a6-0531-4aa7-a2ac-586feed8e40d\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:51.637997 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.637957 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jglzv\" (UniqueName: \"kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv\") pod \"test-curl-7897-140477771908128\" (UID: \"24a917a6-0531-4aa7-a2ac-586feed8e40d\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:51.646943 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.646918 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jglzv\" (UniqueName: \"kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv\") pod \"test-curl-7897-140477771908128\" (UID: \"24a917a6-0531-4aa7-a2ac-586feed8e40d\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:51.755220 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.755122 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:51.905032 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:51.904991 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:51.908412 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:10:51.908385 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24a917a6_0531_4aa7_a2ac_586feed8e40d.slice/crio-ca8a5f6cab534ca1efbb1d5e3de8c85d74d61f7ee29da13457274c41b01a01cc WatchSource:0}: Error finding container ca8a5f6cab534ca1efbb1d5e3de8c85d74d61f7ee29da13457274c41b01a01cc: Status 404 returned error can't find the container with id ca8a5f6cab534ca1efbb1d5e3de8c85d74d61f7ee29da13457274c41b01a01cc Jul 22 12:10:52.067720 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:52.067642 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb54953c-685a-435a-93b2-c3f259abf640" path="/var/lib/kubelet/pods/fb54953c-685a-435a-93b2-c3f259abf640/volumes" Jul 22 12:10:52.102190 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:52.102162 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771877808" Jul 22 12:10:52.102358 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:52.102170 2577 scope.go:117] "RemoveContainer" containerID="b3e82f6bcdae6eba3b4ba59c99c2557cd10d423c0727d0b7b546d4df17c6355a" Jul 22 12:10:52.103616 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:52.103403 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771908128" event={"ID":"24a917a6-0531-4aa7-a2ac-586feed8e40d","Type":"ContainerStarted","Data":"ca8a5f6cab534ca1efbb1d5e3de8c85d74d61f7ee29da13457274c41b01a01cc"} Jul 22 12:10:52.340106 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:52.340015 2577 scope.go:117] "RemoveContainer" containerID="5c9ef8efc465ad9f2584a9027e008aae35af85883959953ae1415a9e26eed10f" Jul 22 12:10:53.110482 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:53.110446 2577 generic.go:356] "Generic (PLEG): container finished" podID="24a917a6-0531-4aa7-a2ac-586feed8e40d" containerID="174a02d300acd6ea15bdd5d7c6f4f64d3b982f6562f3559314236f14ea0aa9c3" exitCode=0 Jul 22 12:10:53.110666 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:53.110549 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771908128" event={"ID":"24a917a6-0531-4aa7-a2ac-586feed8e40d","Type":"ContainerDied","Data":"174a02d300acd6ea15bdd5d7c6f4f64d3b982f6562f3559314236f14ea0aa9c3"} Jul 22 12:10:54.251343 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.251314 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.273072 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.273044 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771908128_24a917a6-0531-4aa7-a2ac-586feed8e40d/test-curl-7897-140477771908128/0.log" Jul 22 12:10:54.301035 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.301005 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:54.303211 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.303183 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:54.363464 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.363429 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jglzv\" (UniqueName: \"kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv\") pod \"24a917a6-0531-4aa7-a2ac-586feed8e40d\" (UID: \"24a917a6-0531-4aa7-a2ac-586feed8e40d\") " Jul 22 12:10:54.365737 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.365674 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv" (OuterVolumeSpecName: "kube-api-access-jglzv") pod "24a917a6-0531-4aa7-a2ac-586feed8e40d" (UID: "24a917a6-0531-4aa7-a2ac-586feed8e40d"). InnerVolumeSpecName "kube-api-access-jglzv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:10:54.427745 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.427710 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:54.428092 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.428079 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a917a6-0531-4aa7-a2ac-586feed8e40d" containerName="test-curl-7897-140477771908128" Jul 22 12:10:54.428132 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.428094 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a917a6-0531-4aa7-a2ac-586feed8e40d" containerName="test-curl-7897-140477771908128" Jul 22 12:10:54.428168 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.428149 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a917a6-0531-4aa7-a2ac-586feed8e40d" containerName="test-curl-7897-140477771908128" Jul 22 12:10:54.431048 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.431033 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.437125 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.437099 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:54.464768 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.464735 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5zh6\" (UniqueName: \"kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6\") pod \"test-curl-7897-140477771908128\" (UID: \"557331b3-636f-4f7a-90d0-c46b506b12f8\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.464914 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.464884 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jglzv\" (UniqueName: \"kubernetes.io/projected/24a917a6-0531-4aa7-a2ac-586feed8e40d-kube-api-access-jglzv\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:10:54.566006 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.565966 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5zh6\" (UniqueName: \"kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6\") pod \"test-curl-7897-140477771908128\" (UID: \"557331b3-636f-4f7a-90d0-c46b506b12f8\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.574770 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.574745 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5zh6\" (UniqueName: \"kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6\") pod \"test-curl-7897-140477771908128\" (UID: \"557331b3-636f-4f7a-90d0-c46b506b12f8\") " pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.741187 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.741097 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:54.876029 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:54.875995 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:54.878964 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:10:54.878939 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod557331b3_636f_4f7a_90d0_c46b506b12f8.slice/crio-1a0b4880f81808c62a5b46fb115e32343ad8494d1ef3d7ef66682d48d6aaa56f WatchSource:0}: Error finding container 1a0b4880f81808c62a5b46fb115e32343ad8494d1ef3d7ef66682d48d6aaa56f: Status 404 returned error can't find the container with id 1a0b4880f81808c62a5b46fb115e32343ad8494d1ef3d7ef66682d48d6aaa56f Jul 22 12:10:55.121553 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:55.121514 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771908128" event={"ID":"557331b3-636f-4f7a-90d0-c46b506b12f8","Type":"ContainerStarted","Data":"1a0b4880f81808c62a5b46fb115e32343ad8494d1ef3d7ef66682d48d6aaa56f"} Jul 22 12:10:55.122800 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:55.122778 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca8a5f6cab534ca1efbb1d5e3de8c85d74d61f7ee29da13457274c41b01a01cc" Jul 22 12:10:55.122800 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:55.122788 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:55.134245 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:55.134214 2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7897-140477771908128" oldPodUID="24a917a6-0531-4aa7-a2ac-586feed8e40d" podUID="557331b3-636f-4f7a-90d0-c46b506b12f8" Jul 22 12:10:56.066956 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:56.066918 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a917a6-0531-4aa7-a2ac-586feed8e40d" path="/var/lib/kubelet/pods/24a917a6-0531-4aa7-a2ac-586feed8e40d/volumes" Jul 22 12:10:56.129525 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:56.129490 2577 generic.go:356] "Generic (PLEG): container finished" podID="557331b3-636f-4f7a-90d0-c46b506b12f8" containerID="9206afd6cfae74c03871e088108bbdd8bdc4e8adee9fc1c928f7b88ab0d5d458" exitCode=0 Jul 22 12:10:56.129706 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:56.129569 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771908128" event={"ID":"557331b3-636f-4f7a-90d0-c46b506b12f8","Type":"ContainerDied","Data":"9206afd6cfae74c03871e088108bbdd8bdc4e8adee9fc1c928f7b88ab0d5d458"} Jul 22 12:10:57.270378 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.270350 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:57.290641 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.290614 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771908128_557331b3-636f-4f7a-90d0-c46b506b12f8/test-curl-7897-140477771908128/0.log" Jul 22 12:10:57.316188 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.316152 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:57.318199 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.318173 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771908128"] Jul 22 12:10:57.391693 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.391614 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5zh6\" (UniqueName: \"kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6\") pod \"557331b3-636f-4f7a-90d0-c46b506b12f8\" (UID: \"557331b3-636f-4f7a-90d0-c46b506b12f8\") " Jul 22 12:10:57.393741 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.393711 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6" (OuterVolumeSpecName: "kube-api-access-w5zh6") pod "557331b3-636f-4f7a-90d0-c46b506b12f8" (UID: "557331b3-636f-4f7a-90d0-c46b506b12f8"). InnerVolumeSpecName "kube-api-access-w5zh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:10:57.440701 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.440664 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:10:57.441084 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.441067 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="557331b3-636f-4f7a-90d0-c46b506b12f8" containerName="test-curl-7897-140477771908128" Jul 22 12:10:57.441179 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.441088 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="557331b3-636f-4f7a-90d0-c46b506b12f8" containerName="test-curl-7897-140477771908128" Jul 22 12:10:57.441237 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.441178 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="557331b3-636f-4f7a-90d0-c46b506b12f8" containerName="test-curl-7897-140477771908128" Jul 22 12:10:57.444055 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.444036 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:10:57.448939 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.448912 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:10:57.492485 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.492451 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6s7v\" (UniqueName: \"kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v\") pod \"test-curl-7897-140477771909136\" (UID: \"c687158e-1403-4e5f-a245-f5a467a9b01f\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:10:57.492672 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.492510 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5zh6\" (UniqueName: \"kubernetes.io/projected/557331b3-636f-4f7a-90d0-c46b506b12f8-kube-api-access-w5zh6\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:10:57.593631 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.593584 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6s7v\" (UniqueName: \"kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v\") pod \"test-curl-7897-140477771909136\" (UID: \"c687158e-1403-4e5f-a245-f5a467a9b01f\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:10:57.601991 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.601962 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6s7v\" (UniqueName: \"kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v\") pod \"test-curl-7897-140477771909136\" (UID: \"c687158e-1403-4e5f-a245-f5a467a9b01f\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:10:57.754579 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.754495 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:10:57.888069 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:57.888044 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:10:57.889777 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:10:57.889749 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc687158e_1403_4e5f_a245_f5a467a9b01f.slice/crio-c081faad941aaab4a5f9512d6cc9057174c9255b697f051ccce820ee528b9d82 WatchSource:0}: Error finding container c081faad941aaab4a5f9512d6cc9057174c9255b697f051ccce820ee528b9d82: Status 404 returned error can't find the container with id c081faad941aaab4a5f9512d6cc9057174c9255b697f051ccce820ee528b9d82 Jul 22 12:10:58.068445 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:58.068362 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="557331b3-636f-4f7a-90d0-c46b506b12f8" path="/var/lib/kubelet/pods/557331b3-636f-4f7a-90d0-c46b506b12f8/volumes" Jul 22 12:10:58.142310 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:58.142271 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771909136" event={"ID":"c687158e-1403-4e5f-a245-f5a467a9b01f","Type":"ContainerStarted","Data":"c081faad941aaab4a5f9512d6cc9057174c9255b697f051ccce820ee528b9d82"} Jul 22 12:10:58.143419 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:58.143401 2577 scope.go:117] "RemoveContainer" containerID="9206afd6cfae74c03871e088108bbdd8bdc4e8adee9fc1c928f7b88ab0d5d458" Jul 22 12:10:58.143508 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:58.143409 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771908128" Jul 22 12:10:59.151140 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:59.151106 2577 generic.go:356] "Generic (PLEG): container finished" podID="c687158e-1403-4e5f-a245-f5a467a9b01f" containerID="edf6481eea95869f826bf52e54af22d6ec9dbeba2452e911a4b779d18e6e395d" exitCode=0 Jul 22 12:10:59.151501 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:10:59.151156 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771909136" event={"ID":"c687158e-1403-4e5f-a245-f5a467a9b01f","Type":"ContainerDied","Data":"edf6481eea95869f826bf52e54af22d6ec9dbeba2452e911a4b779d18e6e395d"} Jul 22 12:11:00.297076 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.297047 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.318856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.318836 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771909136_c687158e-1403-4e5f-a245-f5a467a9b01f/test-curl-7897-140477771909136/0.log" Jul 22 12:11:00.343361 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.343331 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:00.347799 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.347774 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:00.422667 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.422566 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6s7v\" (UniqueName: \"kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v\") pod \"c687158e-1403-4e5f-a245-f5a467a9b01f\" (UID: \"c687158e-1403-4e5f-a245-f5a467a9b01f\") " Jul 22 12:11:00.424750 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.424727 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v" (OuterVolumeSpecName: "kube-api-access-d6s7v") pod "c687158e-1403-4e5f-a245-f5a467a9b01f" (UID: "c687158e-1403-4e5f-a245-f5a467a9b01f"). InnerVolumeSpecName "kube-api-access-d6s7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:11:00.474816 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.474782 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:00.475167 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.475154 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c687158e-1403-4e5f-a245-f5a467a9b01f" containerName="test-curl-7897-140477771909136" Jul 22 12:11:00.475206 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.475170 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c687158e-1403-4e5f-a245-f5a467a9b01f" containerName="test-curl-7897-140477771909136" Jul 22 12:11:00.475251 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.475232 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c687158e-1403-4e5f-a245-f5a467a9b01f" containerName="test-curl-7897-140477771909136" Jul 22 12:11:00.478107 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.478092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.484083 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.484059 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:00.524137 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.524107 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hspd\" (UniqueName: \"kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd\") pod \"test-curl-7897-140477771909136\" (UID: \"d5bc5833-0a2d-4f29-a0a1-49e003302a8b\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.524299 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.524218 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6s7v\" (UniqueName: \"kubernetes.io/projected/c687158e-1403-4e5f-a245-f5a467a9b01f-kube-api-access-d6s7v\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:00.625571 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.625533 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hspd\" (UniqueName: \"kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd\") pod \"test-curl-7897-140477771909136\" (UID: \"d5bc5833-0a2d-4f29-a0a1-49e003302a8b\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.634842 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.634817 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hspd\" (UniqueName: \"kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd\") pod \"test-curl-7897-140477771909136\" (UID: \"d5bc5833-0a2d-4f29-a0a1-49e003302a8b\") " pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.789071 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.788985 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:00.923769 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:00.923742 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:00.925588 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:11:00.925565 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5bc5833_0a2d_4f29_a0a1_49e003302a8b.slice/crio-8a19df935029c13b0a36e47e945f8ea7e1ec3b2c4fab4b65d8a014411a422160 WatchSource:0}: Error finding container 8a19df935029c13b0a36e47e945f8ea7e1ec3b2c4fab4b65d8a014411a422160: Status 404 returned error can't find the container with id 8a19df935029c13b0a36e47e945f8ea7e1ec3b2c4fab4b65d8a014411a422160 Jul 22 12:11:01.165740 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:01.165710 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:01.165898 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:01.165749 2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c081faad941aaab4a5f9512d6cc9057174c9255b697f051ccce820ee528b9d82" Jul 22 12:11:01.166941 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:01.166908 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771909136" event={"ID":"d5bc5833-0a2d-4f29-a0a1-49e003302a8b","Type":"ContainerStarted","Data":"8a19df935029c13b0a36e47e945f8ea7e1ec3b2c4fab4b65d8a014411a422160"} Jul 22 12:11:01.177101 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:01.177074 2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7897-140477771909136" oldPodUID="c687158e-1403-4e5f-a245-f5a467a9b01f" podUID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" Jul 22 12:11:02.069455 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:02.069420 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c687158e-1403-4e5f-a245-f5a467a9b01f" path="/var/lib/kubelet/pods/c687158e-1403-4e5f-a245-f5a467a9b01f/volumes" Jul 22 12:11:02.174240 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:02.174203 2577 generic.go:356] "Generic (PLEG): container finished" podID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" containerID="f31ff282ec56abd42fdea67e76a88923f77a3b5805f445134eb6959b50168ce7" exitCode=0 Jul 22 12:11:02.174417 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:02.174286 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7897-140477771909136" event={"ID":"d5bc5833-0a2d-4f29-a0a1-49e003302a8b","Type":"ContainerDied","Data":"f31ff282ec56abd42fdea67e76a88923f77a3b5805f445134eb6959b50168ce7"} Jul 22 12:11:03.332404 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.332380 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:03.355285 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.355255 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7897-140477771909136_d5bc5833-0a2d-4f29-a0a1-49e003302a8b/test-curl-7897-140477771909136/0.log" Jul 22 12:11:03.378703 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.378673 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:03.382880 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.382854 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7897-140477771909136"] Jul 22 12:11:03.451678 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.451570 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hspd\" (UniqueName: \"kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd\") pod \"d5bc5833-0a2d-4f29-a0a1-49e003302a8b\" (UID: \"d5bc5833-0a2d-4f29-a0a1-49e003302a8b\") " Jul 22 12:11:03.453864 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.453841 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd" (OuterVolumeSpecName: "kube-api-access-4hspd") pod "d5bc5833-0a2d-4f29-a0a1-49e003302a8b" (UID: "d5bc5833-0a2d-4f29-a0a1-49e003302a8b"). InnerVolumeSpecName "kube-api-access-4hspd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:11:03.552802 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:03.552755 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hspd\" (UniqueName: \"kubernetes.io/projected/d5bc5833-0a2d-4f29-a0a1-49e003302a8b-kube-api-access-4hspd\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:04.073752 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:04.069549 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" path="/var/lib/kubelet/pods/d5bc5833-0a2d-4f29-a0a1-49e003302a8b/volumes" Jul 22 12:11:04.186262 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:04.186206 2577 scope.go:117] "RemoveContainer" containerID="f31ff282ec56abd42fdea67e76a88923f77a3b5805f445134eb6959b50168ce7" Jul 22 12:11:04.186447 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:04.186245 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7897-140477771909136" Jul 22 12:11:36.558896 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.558861 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:11:36.559419 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.559066 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" podUID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" containerName="maas-api" containerID="cri-o://af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8" gracePeriod=30 Jul 22 12:11:36.807687 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.807656 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:11:36.841263 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.841184 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") pod \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " Jul 22 12:11:36.841429 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.841278 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca\") pod \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " Jul 22 12:11:36.841429 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.841305 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp8pd\" (UniqueName: \"kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd\") pod \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\" (UID: \"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a\") " Jul 22 12:11:36.841631 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.841589 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" (UID: "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:11:36.843553 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.843525 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" (UID: "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 12:11:36.843691 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.843584 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd" (OuterVolumeSpecName: "kube-api-access-dp8pd") pod "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" (UID: "a7cf5b8a-79dd-44a4-b5f1-f7738d98669a"). InnerVolumeSpecName "kube-api-access-dp8pd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:11:36.942686 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.942650 2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-maas-api-tls\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:36.942686 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.942680 2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-openshift-service-ca\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:36.942686 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:36.942693 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp8pd\" (UniqueName: \"kubernetes.io/projected/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a-kube-api-access-dp8pd\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:37.355561 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.355523 2577 generic.go:356] "Generic (PLEG): container finished" podID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" containerID="af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8" exitCode=0 Jul 22 12:11:37.355786 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.355571 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" Jul 22 12:11:37.355786 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.355649 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" event={"ID":"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a","Type":"ContainerDied","Data":"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8"} Jul 22 12:11:37.355786 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.355710 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl" event={"ID":"a7cf5b8a-79dd-44a4-b5f1-f7738d98669a","Type":"ContainerDied","Data":"0f9fdb2bb0aaa07bcf53534c7bc99a628c7bbed760233a38a3da64df59551878"} Jul 22 12:11:37.355786 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.355739 2577 scope.go:117] "RemoveContainer" containerID="af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8" Jul 22 12:11:37.365524 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.365506 2577 scope.go:117] "RemoveContainer" containerID="af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8" Jul 22 12:11:37.365819 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:11:37.365796 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8\": container with ID starting with af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8 not found: ID does not exist" containerID="af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8" Jul 22 12:11:37.365895 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.365826 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8"} err="failed to get container status \"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8\": rpc error: code = NotFound desc = could not find container \"af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8\": container with ID starting with af82e19413e8ac732ce980521f8678b3792354370e144b2f8eb716157055e9f8 not found: ID does not exist" Jul 22 12:11:37.377897 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.377875 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:11:37.382447 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:37.382418 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-ae7383-79c5fd9c6c-5s7hl"] Jul 22 12:11:38.067481 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:38.067441 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" path="/var/lib/kubelet/pods/a7cf5b8a-79dd-44a4-b5f1-f7738d98669a/volumes" Jul 22 12:11:48.161350 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:48.161309 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:11:48.161865 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:48.161567 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" containerID="cri-o://17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5" gracePeriod=30 Jul 22 12:11:48.876556 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:48.876521 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fcc66564-6ww8x_49bb470b-9c19-4a41-92dc-1ef58f7e2928/authorino/0.log" Jul 22 12:11:49.142494 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:49.142412 2577 patch_prober.go:28] interesting pod/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 12:11:49.142494 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:49.142466 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 12:11:53.411005 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.410976 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:11:53.437531 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.437441 2577 generic.go:356] "Generic (PLEG): container finished" podID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerID="17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5" exitCode=0 Jul 22 12:11:53.437531 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.437520 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" Jul 22 12:11:53.437734 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.437550 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerDied","Data":"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5"} Jul 22 12:11:53.437734 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.437625 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph" event={"ID":"6634b8f0-b094-4902-9934-a8d1fa9ad5b9","Type":"ContainerDied","Data":"53b36359add92ccf794fac252e5bf8db3d69faafc7a85fb36749bc36606ca200"} Jul 22 12:11:53.437734 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.437651 2577 scope.go:117] "RemoveContainer" containerID="17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5" Jul 22 12:11:53.446402 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.446385 2577 scope.go:117] "RemoveContainer" containerID="dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47" Jul 22 12:11:53.454881 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.454857 2577 scope.go:117] "RemoveContainer" containerID="17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5" Jul 22 12:11:53.455160 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:11:53.455135 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5\": container with ID starting with 17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5 not found: ID does not exist" containerID="17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5" Jul 22 12:11:53.455231 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.455168 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5"} err="failed to get container status \"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5\": rpc error: code = NotFound desc = could not find container \"17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5\": container with ID starting with 17a535545a2fdb7dab87862241771fde8855c3eb5d9ccb62355299d0e81960f5 not found: ID does not exist" Jul 22 12:11:53.455231 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.455184 2577 scope.go:117] "RemoveContainer" containerID="dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47" Jul 22 12:11:53.455412 ip-10-0-135-12 kubenswrapper[2577]: E0722 12:11:53.455399 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47\": container with ID starting with dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47 not found: ID does not exist" containerID="dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47" Jul 22 12:11:53.455455 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.455415 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47"} err="failed to get container status \"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47\": rpc error: code = NotFound desc = could not find container \"dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47\": container with ID starting with dcf9c51269c68e18932a2e10b74fd357c83e57d028eebd22fcf8612fed528d47 not found: ID does not exist" Jul 22 12:11:53.487710 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487684 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.487710 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487712 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.487904 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487730 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.487904 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487776 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.487904 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487829 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.487904 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487876 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pjbn\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.488125 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487906 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.488125 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487934 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.488125 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.487994 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:11:53.488274 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488143 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:11:53.488274 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488221 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:11:53.488274 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488220 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 12:11:53.488274 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488236 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token\") pod \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\" (UID: \"6634b8f0-b094-4902-9934-a8d1fa9ad5b9\") " Jul 22 12:11:53.488581 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488563 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:11:53.488792 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488770 2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istiod-ca-cert\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.488856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488799 2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-certs\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.488856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488812 2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-workload-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.488856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488825 2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-credential-socket\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.488856 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.488840 2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-data\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.490104 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.490078 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 12:11:53.490226 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.490170 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 12:11:53.490544 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.490522 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:11:53.490620 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.490579 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn" (OuterVolumeSpecName: "kube-api-access-2pjbn") pod "6634b8f0-b094-4902-9934-a8d1fa9ad5b9" (UID: "6634b8f0-b094-4902-9934-a8d1fa9ad5b9"). InnerVolumeSpecName "kube-api-access-2pjbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 12:11:53.590113 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.590067 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pjbn\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-kube-api-access-2pjbn\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.590113 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.590107 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-token\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.590113 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.590119 2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-envoy\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.590113 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.590127 2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6634b8f0-b094-4902-9934-a8d1fa9ad5b9-istio-podinfo\") on node \"ip-10-0-135-12.ec2.internal\" DevicePath \"\"" Jul 22 12:11:53.761256 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.761228 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:11:53.764988 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:53.764964 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ae7383-openshift-default-68544d954c-bd9ph"] Jul 22 12:11:54.068157 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:54.068077 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" path="/var/lib/kubelet/pods/6634b8f0-b094-4902-9934-a8d1fa9ad5b9/volumes" Jul 22 12:11:55.152480 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:55.152453 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6ctx2_3273357f-2e8d-4a2d-b722-d0161ebe5274/server/0.log" Jul 22 12:11:55.284647 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:55.284618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tdqv6_cf64a448-9766-48b5-b340-c00610a0dcea/manager/0.log" Jul 22 12:11:55.679657 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:55.679629 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f_965bc4fc-6f0e-4cd9-b506-737b2bca2bcf/copy-manifests/0.log" Jul 22 12:11:55.692969 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:55.692944 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f_965bc4fc-6f0e-4cd9-b506-737b2bca2bcf/manager/0.log" Jul 22 12:11:57.246855 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.246832 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/util/0.log" Jul 22 12:11:57.253653 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.253634 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/pull/0.log" Jul 22 12:11:57.260422 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.260407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/extract/0.log" Jul 22 12:11:57.383436 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.383410 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/util/0.log" Jul 22 12:11:57.391169 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.391149 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/pull/0.log" Jul 22 12:11:57.398334 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.398314 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/extract/0.log" Jul 22 12:11:57.519353 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.519274 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/extract/0.log" Jul 22 12:11:57.526127 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.526098 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/util/0.log" Jul 22 12:11:57.533645 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.533623 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/pull/0.log" Jul 22 12:11:57.660378 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.660339 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/extract/0.log" Jul 22 12:11:57.667434 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.667407 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/util/0.log" Jul 22 12:11:57.674753 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.674729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/pull/0.log" Jul 22 12:11:57.803412 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:57.803336 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fcc66564-6ww8x_49bb470b-9c19-4a41-92dc-1ef58f7e2928/authorino/0.log" Jul 22 12:11:58.182992 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:58.182961 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2hhsg_b5898502-f9b7-45e2-af5e-7b04ec574a9d/kuadrant-console-plugin/0.log" Jul 22 12:11:59.450462 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:59.450436 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ffd4cd5bc-wcm8g_aef6707e-8546-4916-88fe-129106ba7f03/kube-auth-proxy/0.log" Jul 22 12:11:59.973687 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:11:59.973657 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5568f487dd-hjg9f_17631881-9aa5-4633-b964-ece76c82b2e1/router/0.log" Jul 22 12:12:00.474806 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:00.474766 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-k99rn_3b90ea5c-1d1b-467c-864a-d9029c4a8b45/main/0.log" Jul 22 12:12:06.751510 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751472 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5cxcj/must-gather-vn2qc"] Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751828 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751840 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751849 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" containerName="maas-api" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751854 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" containerName="maas-api" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751869 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" containerName="test-curl-7897-140477771909136" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751875 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" containerName="test-curl-7897-140477771909136" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751930 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751938 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5bc5833-0a2d-4f29-a0a1-49e003302a8b" containerName="test-curl-7897-140477771909136" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751944 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.751967 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.751970 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7cf5b8a-79dd-44a4-b5f1-f7738d98669a" containerName="maas-api" Jul 22 12:12:06.752317 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.752029 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.752317 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.752035 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6634b8f0-b094-4902-9934-a8d1fa9ad5b9" containerName="istio-proxy" Jul 22 12:12:06.755039 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.755016 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:06.757241 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.757214 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5cxcj\"/\"openshift-service-ca.crt\"" Jul 22 12:12:06.757382 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.757322 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5cxcj\"/\"default-dockercfg-c9d98\"" Jul 22 12:12:06.757915 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.757894 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5cxcj\"/\"kube-root-ca.crt\"" Jul 22 12:12:06.766227 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.766207 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cxcj/must-gather-vn2qc"] Jul 22 12:12:06.897867 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.897833 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/191bdae4-fa0d-4baa-85de-a48561a5aa77-must-gather-output\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:06.897867 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.897869 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c2j8\" (UniqueName: \"kubernetes.io/projected/191bdae4-fa0d-4baa-85de-a48561a5aa77-kube-api-access-9c2j8\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:06.998389 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.998352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/191bdae4-fa0d-4baa-85de-a48561a5aa77-must-gather-output\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:06.998389 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.998393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c2j8\" (UniqueName: \"kubernetes.io/projected/191bdae4-fa0d-4baa-85de-a48561a5aa77-kube-api-access-9c2j8\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:06.998731 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:06.998712 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/191bdae4-fa0d-4baa-85de-a48561a5aa77-must-gather-output\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:07.007962 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:07.007895 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c2j8\" (UniqueName: \"kubernetes.io/projected/191bdae4-fa0d-4baa-85de-a48561a5aa77-kube-api-access-9c2j8\") pod \"must-gather-vn2qc\" (UID: \"191bdae4-fa0d-4baa-85de-a48561a5aa77\") " pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:07.065897 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:07.065867 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" Jul 22 12:12:07.203699 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:07.203676 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cxcj/must-gather-vn2qc"] Jul 22 12:12:07.205279 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:12:07.205254 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod191bdae4_fa0d_4baa_85de_a48561a5aa77.slice/crio-6cee30fb7d56ba99f4c25e5fa1d2b69787cdefa2e4af55867e07402bbe2d9f98 WatchSource:0}: Error finding container 6cee30fb7d56ba99f4c25e5fa1d2b69787cdefa2e4af55867e07402bbe2d9f98: Status 404 returned error can't find the container with id 6cee30fb7d56ba99f4c25e5fa1d2b69787cdefa2e4af55867e07402bbe2d9f98 Jul 22 12:12:07.508929 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:07.508887 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" event={"ID":"191bdae4-fa0d-4baa-85de-a48561a5aa77","Type":"ContainerStarted","Data":"6cee30fb7d56ba99f4c25e5fa1d2b69787cdefa2e4af55867e07402bbe2d9f98"} Jul 22 12:12:10.526206 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:10.526151 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" event={"ID":"191bdae4-fa0d-4baa-85de-a48561a5aa77","Type":"ContainerStarted","Data":"2e566450febee7c1b8dd743d2f41afbccab2cfa525e4fa0f819c978da6407e55"} Jul 22 12:12:11.535965 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:11.535921 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" event={"ID":"191bdae4-fa0d-4baa-85de-a48561a5aa77","Type":"ContainerStarted","Data":"3523132d8e32ca53ccb7358bd5396ff163d44d917f1e7e61db79efba0c914c8d"} Jul 22 12:12:11.552400 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:11.552336 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5cxcj/must-gather-vn2qc" podStartSLOduration=2.402382017 podStartE2EDuration="5.552315723s" totalImagesPullingTime="3.149933706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:12:11.551702913 +0000 UTC m=+2900.059603792" watchObservedRunningTime="2026-07-22 12:12:11.552315723 +0000 UTC m=+2900.060216610" Jul 22 12:12:12.026028 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:12.025996 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5nfx5_b184645a-a72b-4102-9927-a276ab76719d/global-pull-secret-syncer/0.log" Jul 22 12:12:12.222187 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:12.222148 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vkp9f_5d0a01d2-c2e2-4429-bdf4-420358a7c08b/konnectivity-agent/0.log" Jul 22 12:12:12.281864 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:12.281777 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-12.ec2.internal_02a09a085ac5b89656342fe2b159dd5c/haproxy/0.log" Jul 22 12:12:16.790470 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.790379 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/extract/0.log" Jul 22 12:12:16.816380 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.816348 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/util/0.log" Jul 22 12:12:16.852037 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.851992 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mn5f4_1e34d5f6-bc93-4215-a098-d82c3331cd67/pull/0.log" Jul 22 12:12:16.891951 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.891886 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/extract/0.log" Jul 22 12:12:16.917740 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.917703 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/util/0.log" Jul 22 12:12:16.947378 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.947313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mdxfs_e7e62feb-ff53-4e0e-9344-42105dde1e37/pull/0.log" Jul 22 12:12:16.981458 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:16.981425 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/extract/0.log" Jul 22 12:12:17.021385 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.021349 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/util/0.log" Jul 22 12:12:17.051752 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.051662 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73lnsgb_589004d7-85c3-42ad-b4a9-eab9c7cff7cc/pull/0.log" Jul 22 12:12:17.091089 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.091040 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/extract/0.log" Jul 22 12:12:17.122284 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.122253 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/util/0.log" Jul 22 12:12:17.150622 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.150573 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef15hdd7_b273fe00-7a8d-4b05-9a17-d13a51adf853/pull/0.log" Jul 22 12:12:17.638653 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.638618 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-77fcc66564-6ww8x_49bb470b-9c19-4a41-92dc-1ef58f7e2928/authorino/0.log" Jul 22 12:12:17.737516 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:17.737468 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2hhsg_b5898502-f9b7-45e2-af5e-7b04ec574a9d/kuadrant-console-plugin/0.log" Jul 22 12:12:20.046144 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.046092 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-l92nk_4fc79d4d-6476-4c68-8ed8-4716cd450a4b/cluster-monitoring-operator/0.log" Jul 22 12:12:20.085730 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.085694 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-d8z4l_5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0/kube-state-metrics/0.log" Jul 22 12:12:20.104934 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.104907 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-d8z4l_5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0/kube-rbac-proxy-main/0.log" Jul 22 12:12:20.132101 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.132064 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-d8z4l_5d7f67ff-0ac5-4bb6-9f14-6a5a04f572c0/kube-rbac-proxy-self/0.log" Jul 22 12:12:20.191233 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.191204 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-psj7p_3ccb59b6-0c9c-4441-96bc-4197e792fe7b/monitoring-plugin/0.log" Jul 22 12:12:20.311500 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.311409 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jg98n_2819e0f6-29c5-45bc-a6f2-bde8186c324e/node-exporter/0.log" Jul 22 12:12:20.334949 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.334923 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jg98n_2819e0f6-29c5-45bc-a6f2-bde8186c324e/kube-rbac-proxy/0.log" Jul 22 12:12:20.363732 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.363699 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jg98n_2819e0f6-29c5-45bc-a6f2-bde8186c324e/init-textfile/0.log" Jul 22 12:12:20.757503 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.757443 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-5k7s2_5aa846e3-a096-4097-9631-b69c94d5bdd8/prometheus-operator/0.log" Jul 22 12:12:20.777955 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.777926 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-5k7s2_5aa846e3-a096-4097-9631-b69c94d5bdd8/kube-rbac-proxy/0.log" Jul 22 12:12:20.835740 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.835712 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c675b759-6kk74_c38c02c7-6666-4002-8b41-1b7a8e5d8215/telemeter-client/0.log" Jul 22 12:12:20.864094 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.864065 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c675b759-6kk74_c38c02c7-6666-4002-8b41-1b7a8e5d8215/reload/0.log" Jul 22 12:12:20.888048 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:20.888010 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9c675b759-6kk74_c38c02c7-6666-4002-8b41-1b7a8e5d8215/kube-rbac-proxy/0.log" Jul 22 12:12:22.199721 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.199686 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck"] Jul 22 12:12:22.204978 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.204953 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.213433 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.213405 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck"] Jul 22 12:12:22.345647 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.345582 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-podres\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.345985 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.345963 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-proc\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.346183 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.346150 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-lib-modules\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.346349 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.346331 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb797\" (UniqueName: \"kubernetes.io/projected/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-kube-api-access-gb797\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.346522 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.346505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-sys\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.447772 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.447735 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb797\" (UniqueName: \"kubernetes.io/projected/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-kube-api-access-gb797\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.448062 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.448044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-sys\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.448307 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.448215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-sys\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.448460 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.448439 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-podres\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.448750 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.448657 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-podres\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.448882 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.448870 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-proc\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.449057 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.449044 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-lib-modules\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.449314 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.449006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-proc\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.449414 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.449221 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-lib-modules\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.458436 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.458365 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb797\" (UniqueName: \"kubernetes.io/projected/1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0-kube-api-access-gb797\") pod \"perf-node-gather-daemonset-kxnck\" (UID: \"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0\") " pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.519217 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.519186 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:22.706900 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:22.706858 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck"] Jul 22 12:12:22.711338 ip-10-0-135-12 kubenswrapper[2577]: W0722 12:12:22.711292 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1fe8886c_d80d_46b5_beb5_69c2b2ca8ff0.slice/crio-304eaf21e7616122fc6c42ba5a9c9f63bf92cd40f8e0a6685eec433bab20e443 WatchSource:0}: Error finding container 304eaf21e7616122fc6c42ba5a9c9f63bf92cd40f8e0a6685eec433bab20e443: Status 404 returned error can't find the container with id 304eaf21e7616122fc6c42ba5a9c9f63bf92cd40f8e0a6685eec433bab20e443 Jul 22 12:12:23.632425 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:23.632382 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" event={"ID":"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0","Type":"ContainerStarted","Data":"6eae9dfc8522660e45ae0f0a1b7e34be83cbc133f21cf981e9afa28fe62c0bdf"} Jul 22 12:12:23.632930 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:23.632444 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" event={"ID":"1fe8886c-d80d-46b5-beb5-69c2b2ca8ff0","Type":"ContainerStarted","Data":"304eaf21e7616122fc6c42ba5a9c9f63bf92cd40f8e0a6685eec433bab20e443"} Jul 22 12:12:23.632930 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:23.632473 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:23.656857 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:23.656805 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" podStartSLOduration=1.656788101 podStartE2EDuration="1.656788101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:12:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:12:23.655353736 +0000 UTC m=+2912.163254625" watchObservedRunningTime="2026-07-22 12:12:23.656788101 +0000 UTC m=+2912.164688987" Jul 22 12:12:25.371536 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:25.371482 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbzxt_16572c34-03d5-47fa-9182-fbfc27acf6d5/dns/0.log" Jul 22 12:12:25.395409 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:25.395383 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pbzxt_16572c34-03d5-47fa-9182-fbfc27acf6d5/kube-rbac-proxy/0.log" Jul 22 12:12:25.505380 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:25.505352 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dv9cn_20f1f798-5d67-4b30-b583-912ef1c3704e/dns-node-resolver/0.log" Jul 22 12:12:26.126606 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:26.126559 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7875dffb45-85rxj_a347193c-cf7a-402b-8c5f-2c3752edd5f2/registry/0.log" Jul 22 12:12:26.217464 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:26.217430 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w4zp6_16921d4b-0b1f-4d6f-afdd-cc7aa184e095/node-ca/0.log" Jul 22 12:12:27.482452 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:27.482413 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ffd4cd5bc-wcm8g_aef6707e-8546-4916-88fe-129106ba7f03/kube-auth-proxy/0.log" Jul 22 12:12:27.643038 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:27.643008 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5568f487dd-hjg9f_17631881-9aa5-4633-b964-ece76c82b2e1/router/0.log" Jul 22 12:12:28.444120 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:28.444087 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nhdf7_8d624970-f8dc-4111-ba21-c57dca9f579f/serve-healthcheck-canary/0.log" Jul 22 12:12:29.036710 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:29.036682 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z47w_04ee0ab4-1873-4586-94a1-eb901c612e8c/kube-rbac-proxy/0.log" Jul 22 12:12:29.061216 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:29.061184 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z47w_04ee0ab4-1873-4586-94a1-eb901c612e8c/exporter/0.log" Jul 22 12:12:29.088665 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:29.088634 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5z47w_04ee0ab4-1873-4586-94a1-eb901c612e8c/extractor/0.log" Jul 22 12:12:29.650707 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:29.650678 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5cxcj/perf-node-gather-daemonset-kxnck" Jul 22 12:12:31.943950 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:31.943925 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-6ctx2_3273357f-2e8d-4a2d-b722-d0161ebe5274/server/0.log" Jul 22 12:12:32.043030 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:32.043002 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tdqv6_cf64a448-9766-48b5-b340-c00610a0dcea/manager/0.log" Jul 22 12:12:32.292673 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:32.290697 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f_965bc4fc-6f0e-4cd9-b506-737b2bca2bcf/manager/0.log" Jul 22 12:12:32.310144 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:32.310113 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74f4ff9cd9-ljf7f_965bc4fc-6f0e-4cd9-b506-737b2bca2bcf/copy-manifests/0.log" Jul 22 12:12:32.833782 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:32.833751 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ffd4cd5bc-wcm8g_aef6707e-8546-4916-88fe-129106ba7f03/kube-auth-proxy/0.log" Jul 22 12:12:32.948609 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:32.948546 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5568f487dd-hjg9f_17631881-9aa5-4633-b964-ece76c82b2e1/router/0.log" Jul 22 12:12:33.652106 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:33.652026 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55d777d8cf-xlzqb_f5b5be6f-7aa1-4b78-a68e-77894363fba0/manager/0.log" Jul 22 12:12:34.393446 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:34.393373 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ffd4cd5bc-wcm8g_aef6707e-8546-4916-88fe-129106ba7f03/kube-auth-proxy/0.log" Jul 22 12:12:34.503871 ip-10-0-135-12 kubenswrapper[2577]: I0722 12:12:34.503840 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5568f487dd-hjg9f_17631881-9aa5-4633-b964-ece76c82b2e1/router/0.log"